Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 669598 Details for
Bug 886087
Rest query add storage domain fails on fedora18: missing /sbin/scsi_id
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
vdsm log
vdsm.log (text/x-log), 1.50 MB, created by
Ohad Basan
on 2012-12-27 13:41:27 UTC
(
hide
)
Description:
vdsm log
Filename:
MIME Type:
Creator:
Ohad Basan
Created:
2012-12-27 13:41:27 UTC
Size:
1.50 MB
patch
obsolete
>MainThread::INFO::2012-12-04 16:38:42,609::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-04 16:38:44,421::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-04 16:38:44,421::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-04 16:38:44,484::multipath::110::Storage.Multipath::(isEnabled) multipath Defaulting to False >MainThread::DEBUG::2012-12-04 16:38:44,485::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/cp /tmp/tmpkh9WmU /etc/multipath.conf' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,584::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:38:44,584::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath -F' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,613::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ''; <rc> = 1 >MainThread::DEBUG::2012-12-04 16:38:44,613::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/sbin/service multipathd restart' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,836::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = 'Redirecting to /bin/systemctl restart multipathd.service\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:38:44,837::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,855::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:38:44,856::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:38:44,857::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,874::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:38:44,875::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:38:44,875::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:38:44,875::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,891::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:38:44,891::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:38:44,891::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,908::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:38:44,908::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-04 16:38:44,909::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-04 16:38:44,910::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:38:44,910::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-04 16:38:44,910::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:38:44,910::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,919::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-04 16:38:44,928::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-04 16:38:44,928::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-04 16:38:44,929::supervdsm::103::SuperVdsmProxy::(_killSupervdsm) Could not kill old Super Vdsm [Errno 2] No such file or directory: '/var/run/vdsm/svdsm.pid' >Thread-12::DEBUG::2012-12-04 16:38:44,929::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >Thread-12::DEBUG::2012-12-04 16:38:44,929::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 4bc3f581-3057-4e73-a0c9-3ed656a7c003 1226' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:44,945::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-04 16:38:44,946::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-04 16:38:44,947::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-04 16:38:44,955::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >MainThread::DEBUG::2012-12-04 16:38:45,133::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-04 16:38:45,134::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-04 16:38:45,134::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-04 16:38:45,134::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-04 16:38:45,134::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-04 16:38:45,134::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-04 16:38:45,135::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::INFO::2012-12-04 16:38:47,732::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-04 16:38:47,897::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-04 16:38:47,897::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-04 16:38:47,927::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 2003' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:47,947::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 2003: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-04 16:38:47,947::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-04 16:38:47,947::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 401e969f-066e-42e0-81f2-a79f36d059a5 2334' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:48,083::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-04 16:38:48,083::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-04 16:38:48,083::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-04 16:38:48,083::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-04 16:38:48,084::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-04 16:38:48,084::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-04 16:38:48,085::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-04 16:38:49,952::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-04 16:38:49,958::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-04 16:38:49,958::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-04 16:38:49,959::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:49,979::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:38:49,979::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:38:49,980::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:49,996::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:38:49,996::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:38:49,996::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:38:49,997::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:50,012::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:38:50,012::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:38:50,013::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:50,028::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:38:50,029::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-04 16:38:50,029::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-04 16:38:50,030::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:38:50,030::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-04 16:38:50,030::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:38:50,031::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-04 16:38:50,038::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-04 16:38:50,042::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-04 16:38:50,042::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-04 16:38:50,047::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-04 16:38:50,048::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-04 16:38:50,049::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-04 16:38:50,051::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-04 16:38:51,877::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-04 16:38:51,919::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:38:51,920::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:38:51,920::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:38:51,920::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:38:51,920::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:38:51,920::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:38:51,921::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:38:51,921::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-04 16:38:51,921::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-04 16:38:51,925::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-04 16:38:51,925::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-04 16:38:51,925::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-04 16:38:51,926::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-04 16:40:55,603::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-04 16:40:55,770::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-14::DEBUG::2012-12-04 16:40:55,786::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 16:40:55,786::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-04 16:40:55,801::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 16:40:55,802::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-04 16:40:55,817::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 16:40:55,817::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-04 16:40:55,832::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 16:40:55,832::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-14::DEBUG::2012-12-04 16:40:55,847::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 16:40:56,628::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >Thread-15::DEBUG::2012-12-04 16:40:56,877::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-15::DEBUG::2012-12-04 16:40:56,894::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-15::DEBUG::2012-12-04 16:40:56,910::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 16:40:56,910::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-15::DEBUG::2012-12-04 16:40:56,925::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 16:40:56,926::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-15::DEBUG::2012-12-04 16:40:56,941::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 16:40:56,941::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-15::DEBUG::2012-12-04 16:40:56,956::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 16:40:56,956::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-15::DEBUG::2012-12-04 16:40:56,971::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 16:40:56,972::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '10.35.18.155'}} >MainThread::INFO::2012-12-04 16:43:41,187::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-04 16:43:41,188::task::588::TaskManager.Task::(_updateState) Task=`04bab031-a9d9-4480-91dc-f8b186bc40f5`::moving from state init -> state preparing >MainThread::INFO::2012-12-04 16:43:41,188::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-04 16:43:41,189::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-04 16:43:41,214::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-04 16:43:41,215::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-04 16:43:41,215::task::1172::TaskManager.Task::(prepare) Task=`04bab031-a9d9-4480-91dc-f8b186bc40f5`::finished: None >MainThread::DEBUG::2012-12-04 16:43:41,215::task::588::TaskManager.Task::(_updateState) Task=`04bab031-a9d9-4480-91dc-f8b186bc40f5`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-04 16:43:41,215::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-04 16:43:41,215::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-04 16:43:41,215::task::978::TaskManager.Task::(_decref) Task=`04bab031-a9d9-4480-91dc-f8b186bc40f5`::ref 0 aborting False >MainThread::INFO::2012-12-04 16:43:41,216::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-04 16:43:41,216::vdsm::78::vds::(run) <_MainThread(MainThread, started 140634175485760)> >MainThread::INFO::2012-12-04 16:43:41,216::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140633945048832)> >MainThread::INFO::2012-12-04 16:43:41,216::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140633890502400)> >MainThread::INFO::2012-12-04 16:43:41,216::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140633898895104)> >MainThread::INFO::2012-12-04 16:43:41,216::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140632924350208)> >MainThread::INFO::2012-12-04 16:43:41,216::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140633519937280)> >MainThread::INFO::2012-12-04 16:43:41,216::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140633882109696)> >MainThread::INFO::2012-12-04 16:43:41,217::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140633528329984)> >MainThread::INFO::2012-12-04 16:43:41,217::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140633915680512)> >MainThread::INFO::2012-12-04 16:43:41,217::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-04 16:43:41,217::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140632907564800)> >MainThread::INFO::2012-12-04 16:43:41,217::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140633536722688)> >MainThread::INFO::2012-12-04 16:43:41,217::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140633511544576)> >MainThread::INFO::2012-12-04 16:43:41,217::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140633503151872)> >MainThread::INFO::2012-12-04 16:43:41,217::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140633907287808)> >MainThread::INFO::2012-12-04 16:43:42,051::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-04 16:43:42,213::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-04 16:43:42,214::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-04 16:43:42,240::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 2356' (cwd None) >MainThread::DEBUG::2012-12-04 16:43:42,249::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 2356: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-04 16:43:42,250::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-04 16:43:42,250::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 39c75486-40fd-4299-983c-59e24e132413 3200' (cwd None) >MainThread::DEBUG::2012-12-04 16:43:42,383::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-04 16:43:42,383::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-04 16:43:42,383::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-04 16:43:42,384::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-04 16:43:42,384::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-04 16:43:42,384::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-04 16:43:42,385::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-04 16:43:44,254::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-04 16:43:44,260::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-04 16:43:44,261::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-04 16:43:44,261::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-04 16:43:44,280::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:43:44,280::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:43:44,281::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-04 16:43:44,297::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:43:44,298::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:43:44,298::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:43:44,298::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-04 16:43:44,313::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:43:44,313::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:43:44,313::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-04 16:43:44,328::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:43:44,328::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-04 16:43:44,329::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-04 16:43:44,330::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:43:44,330::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-04 16:43:44,330::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:43:44,330::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-04 16:43:44,338::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-04 16:43:44,342::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-04 16:43:44,342::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-04 16:43:44,347::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-04 16:43:44,348::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-04 16:43:44,350::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-04 16:43:44,351::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-04 16:43:46,176::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-04 16:43:46,220::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:43:46,220::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:43:46,221::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:43:46,221::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:43:46,221::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:43:46,221::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:43:46,221::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:43:46,222::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-04 16:43:46,222::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-04 16:43:46,225::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-04 16:43:46,226::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-04 16:43:46,226::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-04 16:43:46,226::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-04 16:44:04,375::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-04 16:44:04,376::task::588::TaskManager.Task::(_updateState) Task=`10b83d4f-2d5c-4697-8579-14a66fc9d2ba`::moving from state init -> state preparing >MainThread::INFO::2012-12-04 16:44:04,376::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-04 16:44:04,377::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-04 16:44:04,399::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-04 16:44:04,399::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-04 16:44:04,399::task::1172::TaskManager.Task::(prepare) Task=`10b83d4f-2d5c-4697-8579-14a66fc9d2ba`::finished: None >MainThread::DEBUG::2012-12-04 16:44:04,399::task::588::TaskManager.Task::(_updateState) Task=`10b83d4f-2d5c-4697-8579-14a66fc9d2ba`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-04 16:44:04,399::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-04 16:44:04,399::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-04 16:44:04,399::task::978::TaskManager.Task::(_decref) Task=`10b83d4f-2d5c-4697-8579-14a66fc9d2ba`::ref 0 aborting False >MainThread::INFO::2012-12-04 16:44:04,400::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-04 16:44:04,400::vdsm::78::vds::(run) <_MainThread(MainThread, started 140497746188096)> >MainThread::INFO::2012-12-04 16:44:04,400::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140497515751168)> >MainThread::INFO::2012-12-04 16:44:04,400::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-04 16:44:04,400::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140496475244288)> >MainThread::INFO::2012-12-04 16:44:04,400::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140497003722496)> >MainThread::INFO::2012-12-04 16:44:04,400::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140497402185472)> >MainThread::INFO::2012-12-04 16:44:04,400::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140497385400064)> >MainThread::INFO::2012-12-04 16:44:04,400::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140497028900608)> >MainThread::INFO::2012-12-04 16:44:04,401::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140497410578176)> >MainThread::INFO::2012-12-04 16:44:04,401::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140497418970880)> >MainThread::INFO::2012-12-04 16:44:04,401::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140497012115200)> >MainThread::INFO::2012-12-04 16:44:04,401::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140497037293312)> >MainThread::INFO::2012-12-04 16:44:04,401::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140496492029696)> >MainThread::INFO::2012-12-04 16:44:04,401::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140497020507904)> >MainThread::INFO::2012-12-04 16:44:04,401::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140497393792768)> >MainThread::INFO::2012-12-04 16:44:05,557::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-04 16:44:05,719::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-04 16:44:05,720::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-04 16:44:05,741::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 3221' (cwd None) >MainThread::DEBUG::2012-12-04 16:44:05,750::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 3221: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-04 16:44:05,750::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-04 16:44:05,751::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 2a763a8c-f90f-4cd8-9113-e09f94caa804 3649' (cwd None) >MainThread::DEBUG::2012-12-04 16:44:05,884::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-04 16:44:05,884::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-04 16:44:05,884::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-04 16:44:05,884::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-04 16:44:05,884::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-04 16:44:05,885::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-04 16:44:05,885::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-04 16:44:07,755::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-04 16:44:07,761::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-04 16:44:07,762::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-04 16:44:07,762::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-04 16:44:07,780::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:44:07,781::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:44:07,781::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-04 16:44:07,796::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:44:07,797::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:44:07,797::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:44:07,797::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-04 16:44:07,812::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:44:07,812::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:44:07,812::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-04 16:44:07,827::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:44:07,827::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-04 16:44:07,828::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-04 16:44:07,829::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:44:07,829::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-04 16:44:07,829::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:44:07,829::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-04 16:44:07,838::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-04 16:44:07,841::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-04 16:44:07,842::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-04 16:44:07,848::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-04 16:44:07,848::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-04 16:44:07,849::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-04 16:44:07,851::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-04 16:44:09,675::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-04 16:44:09,719::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:44:09,720::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:44:09,720::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:44:09,720::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:44:09,720::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:44:09,721::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:44:09,721::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:44:09,721::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-04 16:44:09,721::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-04 16:44:09,725::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-04 16:44:09,725::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-04 16:44:09,725::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-04 16:44:09,726::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-04 16:45:28,943::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-04 16:45:28,944::task::588::TaskManager.Task::(_updateState) Task=`4481efb9-cd54-4eff-b712-b77aa111e717`::moving from state init -> state preparing >MainThread::INFO::2012-12-04 16:45:28,945::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-04 16:45:28,945::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-04 16:45:28,970::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-04 16:45:28,970::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-04 16:45:28,970::task::1172::TaskManager.Task::(prepare) Task=`4481efb9-cd54-4eff-b712-b77aa111e717`::finished: None >MainThread::DEBUG::2012-12-04 16:45:28,971::task::588::TaskManager.Task::(_updateState) Task=`4481efb9-cd54-4eff-b712-b77aa111e717`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-04 16:45:28,971::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-04 16:45:28,971::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-04 16:45:28,971::task::978::TaskManager.Task::(_decref) Task=`4481efb9-cd54-4eff-b712-b77aa111e717`::ref 0 aborting False >MainThread::INFO::2012-12-04 16:45:28,971::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-04 16:45:28,971::vdsm::78::vds::(run) <_MainThread(MainThread, started 140383180314432)> >MainThread::INFO::2012-12-04 16:45:28,971::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140382949877504)> >MainThread::INFO::2012-12-04 16:45:28,972::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140382549571328)> >MainThread::INFO::2012-12-04 16:45:28,972::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140382507607808)> >MainThread::INFO::2012-12-04 16:45:28,972::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140382524393216)> >MainThread::INFO::2012-12-04 16:45:28,972::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140381995915008)> >MainThread::INFO::2012-12-04 16:45:28,972::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140382895331072)> >MainThread::INFO::2012-12-04 16:45:28,972::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140382541178624)> >MainThread::INFO::2012-12-04 16:45:28,972::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-04 16:45:28,972::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140381979129600)> >MainThread::INFO::2012-12-04 16:45:28,973::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140382912116480)> >MainThread::INFO::2012-12-04 16:45:28,973::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140382920509184)> >MainThread::INFO::2012-12-04 16:45:28,973::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140382516000512)> >MainThread::INFO::2012-12-04 16:45:28,973::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140382532785920)> >MainThread::INFO::2012-12-04 16:45:28,973::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140382903723776)> >MainThread::INFO::2012-12-04 16:45:30,210::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-04 16:45:30,372::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-04 16:45:30,372::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-04 16:45:30,396::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 3672' (cwd None) >MainThread::DEBUG::2012-12-04 16:45:30,405::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 3672: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-04 16:45:30,405::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-04 16:45:30,406::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py f8ced60e-5ea3-446c-95bd-5490761a8f97 4114' (cwd None) >MainThread::DEBUG::2012-12-04 16:45:30,539::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-04 16:45:30,539::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-04 16:45:30,539::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-04 16:45:30,539::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-04 16:45:30,539::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-04 16:45:30,540::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-04 16:45:30,541::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-04 16:45:32,410::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-04 16:45:32,415::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-04 16:45:32,416::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-04 16:45:32,417::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-04 16:45:32,436::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:45:32,436::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:45:32,437::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-04 16:45:32,452::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:45:32,453::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:45:32,453::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:45:32,453::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-04 16:45:32,468::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:45:32,468::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:45:32,469::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-04 16:45:32,483::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:45:32,483::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-12::DEBUG::2012-12-04 16:45:32,483::misc::1055::SamplingMethod::(__call__) Got in to sampling method >MainThread::INFO::2012-12-04 16:45:32,484::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-04 16:45:32,485::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-04 16:45:32,485::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:45:32,485::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-04 16:45:32,493::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-04 16:45:32,496::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-04 16:45:32,497::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-04 16:45:32,504::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-04 16:45:32,505::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-04 16:45:32,505::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-04 16:45:32,508::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-04 16:45:34,331::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-04 16:45:34,373::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:45:34,373::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:45:34,373::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:45:34,373::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:45:34,374::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:45:34,374::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:45:34,374::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:45:34,374::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-04 16:45:34,374::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-04 16:45:34,378::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-04 16:45:34,378::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-04 16:45:34,378::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-04 16:45:34,379::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-04 16:47:09,616::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-04 16:47:09,617::task::588::TaskManager.Task::(_updateState) Task=`76a246e5-778f-4b97-b2b6-7a4fdfe30368`::moving from state init -> state preparing >MainThread::INFO::2012-12-04 16:47:09,617::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-04 16:47:09,617::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-04 16:47:09,632::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-04 16:47:09,632::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-04 16:47:09,632::task::1172::TaskManager.Task::(prepare) Task=`76a246e5-778f-4b97-b2b6-7a4fdfe30368`::finished: None >MainThread::DEBUG::2012-12-04 16:47:09,632::task::588::TaskManager.Task::(_updateState) Task=`76a246e5-778f-4b97-b2b6-7a4fdfe30368`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-04 16:47:09,632::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-04 16:47:09,632::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-04 16:47:09,633::task::978::TaskManager.Task::(_decref) Task=`76a246e5-778f-4b97-b2b6-7a4fdfe30368`::ref 0 aborting False >MainThread::INFO::2012-12-04 16:47:09,633::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 12 other threads... >MainThread::INFO::2012-12-04 16:47:09,633::vdsm::78::vds::(run) <_MainThread(MainThread, started 140188109469504)> >MainThread::INFO::2012-12-04 16:47:09,633::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140187879032576)> >MainThread::INFO::2012-12-04 16:47:09,633::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140187772466944)> >MainThread::INFO::2012-12-04 16:47:09,633::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140187371816704)> >MainThread::INFO::2012-12-04 16:47:09,633::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140187780859648)> >MainThread::INFO::2012-12-04 16:47:09,633::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140187363424000)> >MainThread::INFO::2012-12-04 16:47:09,634::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140187380209408)> >MainThread::INFO::2012-12-04 16:47:09,634::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140186851731200)> >MainThread::INFO::2012-12-04 16:47:09,634::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140187388602112)> >MainThread::INFO::2012-12-04 16:47:09,634::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140187396994816)> >MainThread::INFO::2012-12-04 16:47:09,634::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140187747288832)> >MainThread::INFO::2012-12-04 16:47:09,634::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140187755681536)> >MainThread::INFO::2012-12-04 16:47:09,634::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140187764074240)> >MainThread::INFO::2012-12-04 16:49:22,717::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-04 16:49:24,536::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-04 16:49:24,537::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-04 16:49:24,574::supervdsm::103::SuperVdsmProxy::(_killSupervdsm) Could not kill old Super Vdsm [Errno 2] No such file or directory: '/var/run/vdsm/svdsm.pid' >MainThread::DEBUG::2012-12-04 16:49:24,574::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-04 16:49:24,575::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 8d705c6d-7fd4-487a-be7b-02d1caf2f6a7 1280' (cwd None) >MainThread::DEBUG::2012-12-04 16:49:24,714::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-04 16:49:24,714::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-04 16:49:24,714::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-04 16:49:24,714::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-04 16:49:24,714::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-04 16:49:24,715::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-04 16:49:24,716::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-04 16:49:26,581::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-04 16:49:26,586::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-04 16:49:26,587::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-04 16:49:26,587::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-04 16:49:26,609::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:49:26,609::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:49:26,610::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-04 16:49:26,627::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:49:26,627::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:49:26,628::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 16:49:26,628::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-04 16:49:26,644::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 16:49:26,644::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 16:49:26,644::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-04 16:49:26,660::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:49:26,661::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-12::DEBUG::2012-12-04 16:49:26,661::misc::1055::SamplingMethod::(__call__) Got in to sampling method >MainThread::INFO::2012-12-04 16:49:26,662::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-04 16:49:26,662::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-04 16:49:26,662::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 16:49:26,663::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-04 16:49:26,671::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-04 16:49:26,675::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-04 16:49:26,676::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-04 16:49:26,683::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-04 16:49:26,684::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-04 16:49:26,684::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-04 16:49:26,687::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-04 16:49:28,510::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-04 16:49:28,556::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-04 16:49:28,556::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:49:28,557::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:49:28,557::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:49:28,557::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:49:28,557::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 16:49:28,557::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 16:49:28,558::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-04 16:49:28,558::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-04 16:49:28,561::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-04 16:49:28,561::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-04 16:49:28,562::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-04 16:49:28,562::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-04 17:24:35,042::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-04 17:24:35,043::task::588::TaskManager.Task::(_updateState) Task=`9649b44a-470b-478d-ab11-4ba6ad8f9786`::moving from state init -> state preparing >MainThread::INFO::2012-12-04 17:24:35,044::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-04 17:24:35,044::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-04 17:24:35,068::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-04 17:24:35,069::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-04 17:24:35,069::task::1172::TaskManager.Task::(prepare) Task=`9649b44a-470b-478d-ab11-4ba6ad8f9786`::finished: None >MainThread::DEBUG::2012-12-04 17:24:35,069::task::588::TaskManager.Task::(_updateState) Task=`9649b44a-470b-478d-ab11-4ba6ad8f9786`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-04 17:24:35,069::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-04 17:24:35,069::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-04 17:24:35,069::task::978::TaskManager.Task::(_decref) Task=`9649b44a-470b-478d-ab11-4ba6ad8f9786`::ref 0 aborting False >MainThread::INFO::2012-12-04 17:24:35,069::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 12 other threads... >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <_MainThread(MainThread, started 140337255974720)> >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140337025537792)> >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140336996169472)> >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140336890365696)> >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140336898758400)> >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140336303171328)> >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140336311564032)> >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140336881972992)> >MainThread::INFO::2012-12-04 17:24:35,070::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140336915543808)> >MainThread::INFO::2012-12-04 17:24:35,071::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140336873580288)> >MainThread::INFO::2012-12-04 17:24:35,071::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140336269600512)> >MainThread::INFO::2012-12-04 17:24:35,071::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140336865187584)> >MainThread::INFO::2012-12-04 17:24:35,071::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140336907151104)> >MainThread::INFO::2012-12-04 18:17:09,273::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-04 18:17:10,899::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-04 18:17:10,899::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-04 18:17:10,952::supervdsm::103::SuperVdsmProxy::(_killSupervdsm) Could not kill old Super Vdsm [Errno 2] No such file or directory: '/var/run/vdsm/svdsm.pid' >MainThread::DEBUG::2012-12-04 18:17:10,953::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-04 18:17:10,953::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py c3fab12b-bfd5-47bf-80ad-937c781fc537 1236' (cwd None) >MainThread::DEBUG::2012-12-04 18:17:11,093::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-04 18:17:11,093::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-04 18:17:11,094::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-04 18:17:11,094::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-04 18:17:11,094::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-04 18:17:11,094::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-04 18:17:11,095::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-04 18:17:12,957::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-04 18:17:12,965::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-04 18:17:12,966::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-04 18:17:12,966::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-04 18:17:12,990::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 18:17:12,990::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 18:17:12,991::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-04 18:17:13,007::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 18:17:13,007::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 18:17:13,008::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 18:17:13,008::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-04 18:17:13,023::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 18:17:13,023::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 18:17:13,023::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-04 18:17:13,038::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-04 18:17:13,039::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-04 18:17:13,040::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-04 18:17:13,040::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 18:17:13,040::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-04 18:17:13,041::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 18:17:13,041::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-04 18:17:13,053::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-04 18:17:13,053::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-04 18:17:13,057::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-04 18:17:13,077::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-04 18:17:13,077::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-04 18:17:13,078::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-04 18:17:13,081::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-04 18:17:14,891::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-04 18:17:14,929::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-04 18:17:14,929::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 18:17:14,930::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 18:17:14,930::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 18:17:14,930::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 18:17:14,930::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 18:17:14,930::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 18:17:14,931::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-04 18:17:14,931::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-04 18:17:14,934::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-04 18:17:14,935::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-04 18:17:14,935::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-04 18:17:14,935::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-04 18:41:53,980::BindingXMLRPC::859::vds::(wrapper) client [10.35.148.46]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-04 18:41:54,118::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-14::DEBUG::2012-12-04 18:41:54,135::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:41:54,135::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-04 18:41:54,151::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:41:54,151::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-04 18:41:54,166::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:41:54,167::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-04 18:41:54,182::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:41:54,182::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-14::DEBUG::2012-12-04 18:41:54,197::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:41:54,976::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >MainThread::INFO::2012-12-04 18:42:19,993::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-04 18:42:19,994::task::588::TaskManager.Task::(_updateState) Task=`9b8e76c3-e26e-4dbd-a8d6-8107a0045b24`::moving from state init -> state preparing >MainThread::INFO::2012-12-04 18:42:19,994::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-04 18:42:19,994::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-04 18:42:20,026::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-04 18:42:20,026::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-04 18:42:20,027::task::1172::TaskManager.Task::(prepare) Task=`9b8e76c3-e26e-4dbd-a8d6-8107a0045b24`::finished: None >MainThread::DEBUG::2012-12-04 18:42:20,027::task::588::TaskManager.Task::(_updateState) Task=`9b8e76c3-e26e-4dbd-a8d6-8107a0045b24`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-04 18:42:20,027::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-04 18:42:20,027::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-04 18:42:20,027::task::978::TaskManager.Task::(_decref) Task=`9b8e76c3-e26e-4dbd-a8d6-8107a0045b24`::ref 0 aborting False >MainThread::INFO::2012-12-04 18:42:20,027::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-04 18:42:20,027::vdsm::78::vds::(run) <_MainThread(MainThread, started 139919593609024)> >MainThread::INFO::2012-12-04 18:42:20,028::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139919363172096)> >MainThread::INFO::2012-12-04 18:42:20,028::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-04 18:42:20,028::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139918391097088)> >MainThread::INFO::2012-12-04 18:42:20,028::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139919325411072)> >MainThread::INFO::2012-12-04 18:42:20,028::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139918936360704)> >MainThread::INFO::2012-12-04 18:42:20,028::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139919333803776)> >MainThread::INFO::2012-12-04 18:42:20,028::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139918927968000)> >MainThread::INFO::2012-12-04 18:42:20,028::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139918944753408)> >MainThread::INFO::2012-12-04 18:42:20,029::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139918953146112)> >MainThread::INFO::2012-12-04 18:42:20,029::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139918407882496)> >MainThread::INFO::2012-12-04 18:42:20,029::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139918961538816)> >MainThread::INFO::2012-12-04 18:42:20,029::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139918919575296)> >MainThread::INFO::2012-12-04 18:42:20,029::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139919317018368)> >MainThread::INFO::2012-12-04 18:42:20,029::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139919308625664)> >MainThread::INFO::2012-12-04 18:44:47,785::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-04 18:44:49,823::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-04 18:44:49,823::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-04 18:44:49,864::supervdsm::103::SuperVdsmProxy::(_killSupervdsm) Could not kill old Super Vdsm [Errno 2] No such file or directory: '/var/run/vdsm/svdsm.pid' >MainThread::DEBUG::2012-12-04 18:44:49,864::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-04 18:44:49,864::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 18c18f51-9327-4142-a77b-f3c7ab5869f9 1248' (cwd None) >MainThread::DEBUG::2012-12-04 18:44:50,005::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-04 18:44:50,005::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-04 18:44:50,006::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-04 18:44:50,006::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-04 18:44:50,006::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-04 18:44:50,006::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-04 18:44:50,007::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-04 18:44:51,869::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-04 18:44:51,875::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-04 18:44:51,875::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-04 18:44:51,876::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-04 18:44:51,897::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 18:44:51,897::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 18:44:51,898::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-04 18:44:51,915::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-04 18:44:51,915::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 18:44:51,916::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-04 18:44:51,916::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-04 18:44:51,931::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-04 18:44:51,931::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-04 18:44:51,931::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-04 18:44:51,946::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-04 18:44:51,946::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-04 18:44:51,947::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-04 18:44:51,948::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 18:44:51,948::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-04 18:44:51,948::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-04 18:44:51,949::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-04 18:44:51,956::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-04 18:44:51,959::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-04 18:44:51,959::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-04 18:44:51,968::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-04 18:44:51,969::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-04 18:44:51,971::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-04 18:44:51,972::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-04 18:44:53,803::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-04 18:44:53,847::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-04 18:44:53,847::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 18:44:53,848::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 18:44:53,848::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 18:44:53,848::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 18:44:53,848::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-04 18:44:53,848::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-04 18:44:53,849::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-04 18:44:53,849::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-04 18:44:53,853::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-04 18:44:53,853::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-04 18:44:53,853::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-04 18:44:53,854::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-04 18:47:24,120::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-04 18:47:24,258::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-14::DEBUG::2012-12-04 18:47:24,272::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:47:24,273::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-04 18:47:24,287::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:47:24,288::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-04 18:47:24,302::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:47:24,302::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-04 18:47:24,316::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:47:24,316::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-14::DEBUG::2012-12-04 18:47:24,330::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-04 18:47:25,109::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >Thread-15::DEBUG::2012-12-04 18:47:25,364::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-15::DEBUG::2012-12-04 18:47:25,381::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-15::DEBUG::2012-12-04 18:47:25,396::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 18:47:25,396::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-15::DEBUG::2012-12-04 18:47:25,410::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 18:47:25,411::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-15::DEBUG::2012-12-04 18:47:25,425::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 18:47:25,425::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-15::DEBUG::2012-12-04 18:47:25,438::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 18:47:25,439::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-15::DEBUG::2012-12-04 18:47:25,453::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-04 18:47:25,453::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '10.35.18.155'}} >Thread-17::DEBUG::2012-12-04 18:47:27,738::task::588::TaskManager.Task::(_updateState) Task=`2efc4bc3-a624-4766-be13-a52bf64776c9`::moving from state init -> state preparing >Thread-17::INFO::2012-12-04 18:47:27,738::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-17::INFO::2012-12-04 18:47:27,739::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-17::DEBUG::2012-12-04 18:47:27,739::task::1172::TaskManager.Task::(prepare) Task=`2efc4bc3-a624-4766-be13-a52bf64776c9`::finished: {} >Thread-17::DEBUG::2012-12-04 18:47:27,739::task::588::TaskManager.Task::(_updateState) Task=`2efc4bc3-a624-4766-be13-a52bf64776c9`::moving from state preparing -> state finished >Thread-17::DEBUG::2012-12-04 18:47:27,740::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2012-12-04 18:47:27,740::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2012-12-04 18:47:27,740::task::978::TaskManager.Task::(_decref) Task=`2efc4bc3-a624-4766-be13-a52bf64776c9`::ref 0 aborting False >Thread-23::DEBUG::2012-12-04 18:47:38,384::task::588::TaskManager.Task::(_updateState) Task=`602066d4-80d1-4319-87fa-9dac5237ebe3`::moving from state init -> state preparing >Thread-23::INFO::2012-12-04 18:47:38,385::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-23::INFO::2012-12-04 18:47:38,385::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-23::DEBUG::2012-12-04 18:47:38,385::task::1172::TaskManager.Task::(prepare) Task=`602066d4-80d1-4319-87fa-9dac5237ebe3`::finished: {} >Thread-23::DEBUG::2012-12-04 18:47:38,385::task::588::TaskManager.Task::(_updateState) Task=`602066d4-80d1-4319-87fa-9dac5237ebe3`::moving from state preparing -> state finished >Thread-23::DEBUG::2012-12-04 18:47:38,385::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-23::DEBUG::2012-12-04 18:47:38,385::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-23::DEBUG::2012-12-04 18:47:38,385::task::978::TaskManager.Task::(_decref) Task=`602066d4-80d1-4319-87fa-9dac5237ebe3`::ref 0 aborting False >Thread-25::DEBUG::2012-12-04 18:47:39,129::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-25::DEBUG::2012-12-04 18:47:39,129::task::588::TaskManager.Task::(_updateState) Task=`cd0cebd2-25ab-4f38-9dfb-b831bec44c1b`::moving from state init -> state preparing >Thread-25::INFO::2012-12-04 18:47:39,129::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-25::INFO::2012-12-04 18:47:39,129::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-25::DEBUG::2012-12-04 18:47:39,129::task::1172::TaskManager.Task::(prepare) Task=`cd0cebd2-25ab-4f38-9dfb-b831bec44c1b`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-25::DEBUG::2012-12-04 18:47:39,130::task::588::TaskManager.Task::(_updateState) Task=`cd0cebd2-25ab-4f38-9dfb-b831bec44c1b`::moving from state preparing -> state finished >Thread-25::DEBUG::2012-12-04 18:47:39,130::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-25::DEBUG::2012-12-04 18:47:39,130::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-25::DEBUG::2012-12-04 18:47:39,130::task::978::TaskManager.Task::(_decref) Task=`cd0cebd2-25ab-4f38-9dfb-b831bec44c1b`::ref 0 aborting False >Thread-26::DEBUG::2012-12-04 18:47:39,182::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-26::DEBUG::2012-12-04 18:47:39,183::task::588::TaskManager.Task::(_updateState) Task=`630bec47-5bcc-4bde-856a-b6fddad84314`::moving from state init -> state preparing >Thread-26::INFO::2012-12-04 18:47:39,183::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-26::DEBUG::2012-12-04 18:47:39,214::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604 /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604' (cwd None) >Thread-26::DEBUG::2012-12-04 18:47:39,467::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-26::DEBUG::2012-12-04 18:47:39,467::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-26::DEBUG::2012-12-04 18:47:39,467::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-26::DEBUG::2012-12-04 18:47:39,467::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-26::DEBUG::2012-12-04 18:47:39,467::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-26::DEBUG::2012-12-04 18:47:39,468::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-26::INFO::2012-12-04 18:47:39,468::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-26::DEBUG::2012-12-04 18:47:39,468::task::1172::TaskManager.Task::(prepare) Task=`630bec47-5bcc-4bde-856a-b6fddad84314`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-26::DEBUG::2012-12-04 18:47:39,468::task::588::TaskManager.Task::(_updateState) Task=`630bec47-5bcc-4bde-856a-b6fddad84314`::moving from state preparing -> state finished >Thread-26::DEBUG::2012-12-04 18:47:39,468::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-26::DEBUG::2012-12-04 18:47:39,468::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-26::DEBUG::2012-12-04 18:47:39,469::task::978::TaskManager.Task::(_decref) Task=`630bec47-5bcc-4bde-856a-b6fddad84314`::ref 0 aborting False >Thread-27::DEBUG::2012-12-04 18:47:39,541::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-27::DEBUG::2012-12-04 18:47:39,541::task::588::TaskManager.Task::(_updateState) Task=`9c355b46-0f1c-4c16-837d-a8612073cecb`::moving from state init -> state preparing >Thread-27::INFO::2012-12-04 18:47:39,541::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=1, storageType=1, remotePath='10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', options=None) >Thread-27::DEBUG::2012-12-04 18:47:39,542::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-27::DEBUG::2012-12-04 18:47:39,542::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-27::DEBUG::2012-12-04 18:47:39,542::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-27::DEBUG::2012-12-04 18:47:39,542::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-27::DEBUG::2012-12-04 18:47:39,542::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-27::DEBUG::2012-12-04 18:47:39,552::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-27::DEBUG::2012-12-04 18:47:39,552::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-27::DEBUG::2012-12-04 18:47:41,387::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-27::DEBUG::2012-12-04 18:47:41,430::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-27::DEBUG::2012-12-04 18:47:41,431::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2012-12-04 18:47:41,431::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2012-12-04 18:47:41,431::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2012-12-04 18:47:41,431::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2012-12-04 18:47:41,431::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2012-12-04 18:47:41,432::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2012-12-04 18:47:41,432::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-27::DEBUG::2012-12-04 18:47:41,432::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-27::DEBUG::2012-12-04 18:47:41,433::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-27::DEBUG::2012-12-04 18:47:41,455::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-27::DEBUG::2012-12-04 18:47:41,455::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-27::INFO::2012-12-04 18:47:41,626::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-27::DEBUG::2012-12-04 18:47:41,626::task::1172::TaskManager.Task::(prepare) Task=`9c355b46-0f1c-4c16-837d-a8612073cecb`::finished: {'domlist': []} >Thread-27::DEBUG::2012-12-04 18:47:41,626::task::588::TaskManager.Task::(_updateState) Task=`9c355b46-0f1c-4c16-837d-a8612073cecb`::moving from state preparing -> state finished >Thread-27::DEBUG::2012-12-04 18:47:41,627::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-27::DEBUG::2012-12-04 18:47:41,627::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-27::DEBUG::2012-12-04 18:47:41,627::task::978::TaskManager.Task::(_decref) Task=`9c355b46-0f1c-4c16-837d-a8612073cecb`::ref 0 aborting False >Thread-31::DEBUG::2012-12-04 18:47:41,820::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-31::DEBUG::2012-12-04 18:47:41,820::task::588::TaskManager.Task::(_updateState) Task=`d66d5624-68b8-426e-ab61-f3752841768e`::moving from state init -> state preparing >Thread-31::INFO::2012-12-04 18:47:41,821::logUtils::37::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', domainName='DataDomainRest', typeSpecificArg='10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', domClass=1, domVersion='3', options=None) >Thread-31::DEBUG::2012-12-04 18:47:41,821::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-31::DEBUG::2012-12-04 18:47:41,821::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 8dc966ae-7319-4db8-8fab-33f01a0512b6' (cwd None) >Thread-31::DEBUG::2012-12-04 18:47:41,838::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found\n'; <rc> = 5 >Thread-31::WARNING::2012-12-04 18:47:41,838::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found'] >Thread-31::DEBUG::2012-12-04 18:47:41,839::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-31::INFO::2012-12-04 18:47:41,841::nfsSD::64::Storage.StorageDomain::(create) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 domainName=DataDomainRest remotePath=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604 domClass=1 >Thread-31::DEBUG::2012-12-04 18:47:45,291::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-31::DEBUG::2012-12-04 18:47:45,292::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-31::WARNING::2012-12-04 18:47:45,292::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-31::DEBUG::2012-12-04 18:47:45,292::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-31::DEBUG::2012-12-04 18:47:45,292::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-31::DEBUG::2012-12-04 18:47:45,292::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=0c95bb213492ef395d7beb8c1df2f8cd4d06d5c3'] >Thread-31::DEBUG::2012-12-04 18:47:45,308::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-31::DEBUG::2012-12-04 18:47:45,309::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-31::DEBUG::2012-12-04 18:47:45,310::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-31::DEBUG::2012-12-04 18:47:45,313::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=0c95bb213492ef395d7beb8c1df2f8cd4d06d5c3'] >Thread-31::DEBUG::2012-12-04 18:47:45,316::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-31::DEBUG::2012-12-04 18:47:45,316::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS' >Thread-31::DEBUG::2012-12-04 18:47:45,317::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS' >Thread-31::DEBUG::2012-12-04 18:47:45,357::sd::354::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-31::INFO::2012-12-04 18:47:45,357::logUtils::39::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-31::DEBUG::2012-12-04 18:47:45,357::task::1172::TaskManager.Task::(prepare) Task=`d66d5624-68b8-426e-ab61-f3752841768e`::finished: None >Thread-31::DEBUG::2012-12-04 18:47:45,358::task::588::TaskManager.Task::(_updateState) Task=`d66d5624-68b8-426e-ab61-f3752841768e`::moving from state preparing -> state finished >Thread-31::DEBUG::2012-12-04 18:47:45,358::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-31::DEBUG::2012-12-04 18:47:45,358::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-31::DEBUG::2012-12-04 18:47:45,358::task::978::TaskManager.Task::(_decref) Task=`d66d5624-68b8-426e-ab61-f3752841768e`::ref 0 aborting False >Thread-35::DEBUG::2012-12-04 18:47:45,403::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-35::DEBUG::2012-12-04 18:47:45,403::task::588::TaskManager.Task::(_updateState) Task=`61482c31-8f2e-4a00-86f8-be3c8714f078`::moving from state init -> state preparing >Thread-35::INFO::2012-12-04 18:47:45,403::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', options=None) >Thread-35::DEBUG::2012-12-04 18:47:45,404::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`1f112935-a7c6-45a0-a627-6788da9b5986`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-35::DEBUG::2012-12-04 18:47:45,404::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-35::DEBUG::2012-12-04 18:47:45,404::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-35::DEBUG::2012-12-04 18:47:45,404::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`1f112935-a7c6-45a0-a627-6788da9b5986`::Granted request >Thread-35::DEBUG::2012-12-04 18:47:45,405::task::817::TaskManager.Task::(resourceAcquired) Task=`61482c31-8f2e-4a00-86f8-be3c8714f078`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-35::DEBUG::2012-12-04 18:47:45,405::task::978::TaskManager.Task::(_decref) Task=`61482c31-8f2e-4a00-86f8-be3c8714f078`::ref 1 aborting False >Thread-35::DEBUG::2012-12-04 18:47:45,405::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-35::DEBUG::2012-12-04 18:47:45,405::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 8dc966ae-7319-4db8-8fab-33f01a0512b6' (cwd None) >Thread-35::DEBUG::2012-12-04 18:47:45,422::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found\n'; <rc> = 5 >Thread-35::WARNING::2012-12-04 18:47:45,423::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found'] >Thread-35::DEBUG::2012-12-04 18:47:45,423::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-35::DEBUG::2012-12-04 18:47:45,429::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-35::DEBUG::2012-12-04 18:47:45,429::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-35::DEBUG::2012-12-04 18:47:45,431::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=0c95bb213492ef395d7beb8c1df2f8cd4d06d5c3'] >Thread-35::DEBUG::2012-12-04 18:47:45,433::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-35::WARNING::2012-12-04 18:47:45,433::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS already registered >Thread-35::WARNING::2012-12-04 18:47:45,433::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS already registered >Thread-35::INFO::2012-12-04 18:47:45,434::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '844003016704', 'disktotal': '844003016704', 'mdafree': 0}} >Thread-35::DEBUG::2012-12-04 18:47:45,434::task::1172::TaskManager.Task::(prepare) Task=`61482c31-8f2e-4a00-86f8-be3c8714f078`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '844003016704', 'disktotal': '844003016704', 'mdafree': 0}} >Thread-35::DEBUG::2012-12-04 18:47:45,434::task::588::TaskManager.Task::(_updateState) Task=`61482c31-8f2e-4a00-86f8-be3c8714f078`::moving from state preparing -> state finished >Thread-35::DEBUG::2012-12-04 18:47:45,434::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >Thread-35::DEBUG::2012-12-04 18:47:45,434::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-35::DEBUG::2012-12-04 18:47:45,435::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-35::DEBUG::2012-12-04 18:47:45,435::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-35::DEBUG::2012-12-04 18:47:45,435::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-35::DEBUG::2012-12-04 18:47:45,435::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-35::DEBUG::2012-12-04 18:47:45,435::task::978::TaskManager.Task::(_decref) Task=`61482c31-8f2e-4a00-86f8-be3c8714f078`::ref 0 aborting False >Thread-38::DEBUG::2012-12-04 18:47:46,883::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-38::DEBUG::2012-12-04 18:47:46,884::task::588::TaskManager.Task::(_updateState) Task=`d127b556-d3c8-4e07-a6b6-b2521a0ef335`::moving from state init -> state preparing >Thread-38::INFO::2012-12-04 18:47:46,884::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-38::INFO::2012-12-04 18:47:46,884::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-38::DEBUG::2012-12-04 18:47:46,884::task::1172::TaskManager.Task::(prepare) Task=`d127b556-d3c8-4e07-a6b6-b2521a0ef335`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-38::DEBUG::2012-12-04 18:47:46,884::task::588::TaskManager.Task::(_updateState) Task=`d127b556-d3c8-4e07-a6b6-b2521a0ef335`::moving from state preparing -> state finished >Thread-38::DEBUG::2012-12-04 18:47:46,885::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-38::DEBUG::2012-12-04 18:47:46,885::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-38::DEBUG::2012-12-04 18:47:46,885::task::978::TaskManager.Task::(_decref) Task=`d127b556-d3c8-4e07-a6b6-b2521a0ef335`::ref 0 aborting False >Thread-39::DEBUG::2012-12-04 18:47:46,935::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-39::DEBUG::2012-12-04 18:47:46,935::task::588::TaskManager.Task::(_updateState) Task=`14b82b38-fd8f-465d-989a-2160d8e02ced`::moving from state init -> state preparing >Thread-39::INFO::2012-12-04 18:47:46,935::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-39::DEBUG::2012-12-04 18:47:46,939::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-39::DEBUG::2012-12-04 18:47:46,939::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-39::DEBUG::2012-12-04 18:47:46,939::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-39::DEBUG::2012-12-04 18:47:46,939::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-39::DEBUG::2012-12-04 18:47:46,940::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-39::DEBUG::2012-12-04 18:47:46,940::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-39::INFO::2012-12-04 18:47:46,940::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-39::DEBUG::2012-12-04 18:47:46,940::task::1172::TaskManager.Task::(prepare) Task=`14b82b38-fd8f-465d-989a-2160d8e02ced`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-39::DEBUG::2012-12-04 18:47:46,940::task::588::TaskManager.Task::(_updateState) Task=`14b82b38-fd8f-465d-989a-2160d8e02ced`::moving from state preparing -> state finished >Thread-39::DEBUG::2012-12-04 18:47:46,940::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-39::DEBUG::2012-12-04 18:47:46,940::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-39::DEBUG::2012-12-04 18:47:46,941::task::978::TaskManager.Task::(_decref) Task=`14b82b38-fd8f-465d-989a-2160d8e02ced`::ref 0 aborting False >Thread-40::DEBUG::2012-12-04 18:47:46,994::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-40::DEBUG::2012-12-04 18:47:46,995::task::588::TaskManager.Task::(_updateState) Task=`802d3ff3-6dd3-4b37-8dcd-383610a86255`::moving from state init -> state preparing >Thread-40::INFO::2012-12-04 18:47:46,995::logUtils::37::dispatcher::(wrapper) Run and protect: createStoragePool(poolType=None, spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', poolName='DefaultRest', masterDom='8dc966ae-7319-4db8-8fab-33f01a0512b6', domList=['8dc966ae-7319-4db8-8fab-33f01a0512b6'], masterVersion=1, lockPolicy=None, lockRenewalIntervalSec=5, leaseTimeSec=60, ioOpTimeoutSec=10, leaseRetries=3, options=None) >Thread-40::DEBUG::2012-12-04 18:47:46,995::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-40::DEBUG::2012-12-04 18:47:46,995::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-40::DEBUG::2012-12-04 18:47:46,995::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-40::DEBUG::2012-12-04 18:47:46,996::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-40::DEBUG::2012-12-04 18:47:46,996::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-40::DEBUG::2012-12-04 18:47:47,005::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-40::DEBUG::2012-12-04 18:47:47,005::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-40::DEBUG::2012-12-04 18:47:48,840::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-40::DEBUG::2012-12-04 18:47:48,882::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-40::DEBUG::2012-12-04 18:47:48,883::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-40::DEBUG::2012-12-04 18:47:48,883::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-40::DEBUG::2012-12-04 18:47:48,883::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-40::DEBUG::2012-12-04 18:47:48,883::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-40::DEBUG::2012-12-04 18:47:48,884::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-40::DEBUG::2012-12-04 18:47:48,884::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-40::DEBUG::2012-12-04 18:47:48,884::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-40::DEBUG::2012-12-04 18:47:48,884::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-40::DEBUG::2012-12-04 18:47:48,885::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 8dc966ae-7319-4db8-8fab-33f01a0512b6' (cwd None) >Thread-40::DEBUG::2012-12-04 18:47:48,901::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found\n'; <rc> = 5 >Thread-40::WARNING::2012-12-04 18:47:48,901::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found'] >Thread-40::DEBUG::2012-12-04 18:47:48,902::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-40::DEBUG::2012-12-04 18:47:48,907::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-40::DEBUG::2012-12-04 18:47:48,907::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-40::DEBUG::2012-12-04 18:47:48,910::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=0c95bb213492ef395d7beb8c1df2f8cd4d06d5c3'] >Thread-40::DEBUG::2012-12-04 18:47:48,911::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-40::WARNING::2012-12-04 18:47:48,911::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS already registered >Thread-40::WARNING::2012-12-04 18:47:48,911::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS already registered >Thread-40::INFO::2012-12-04 18:47:48,912::fileSD::260::Storage.StorageDomain::(validate) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-40::DEBUG::2012-12-04 18:47:48,914::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=0c95bb213492ef395d7beb8c1df2f8cd4d06d5c3'] >Thread-40::DEBUG::2012-12-04 18:47:48,915::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`797d85bd-f0f4-4785-96e4-a25ac098440f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-40::DEBUG::2012-12-04 18:47:48,915::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-40::DEBUG::2012-12-04 18:47:48,915::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-40::DEBUG::2012-12-04 18:47:48,915::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`797d85bd-f0f4-4785-96e4-a25ac098440f`::Granted request >Thread-40::DEBUG::2012-12-04 18:47:48,915::task::817::TaskManager.Task::(resourceAcquired) Task=`802d3ff3-6dd3-4b37-8dcd-383610a86255`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (exclusive) >Thread-40::DEBUG::2012-12-04 18:47:48,916::task::978::TaskManager.Task::(_decref) Task=`802d3ff3-6dd3-4b37-8dcd-383610a86255`::ref 1 aborting False >Thread-40::DEBUG::2012-12-04 18:47:48,916::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`10e0775a-4037-48a4-8cfd-758112138fe0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-40::DEBUG::2012-12-04 18:47:48,916::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'exclusive' >Thread-40::DEBUG::2012-12-04 18:47:48,916::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'exclusive' (1 active user) >Thread-40::DEBUG::2012-12-04 18:47:48,916::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`10e0775a-4037-48a4-8cfd-758112138fe0`::Granted request >Thread-40::DEBUG::2012-12-04 18:47:48,917::task::817::TaskManager.Task::(resourceAcquired) Task=`802d3ff3-6dd3-4b37-8dcd-383610a86255`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (exclusive) >Thread-40::DEBUG::2012-12-04 18:47:48,917::task::978::TaskManager.Task::(_decref) Task=`802d3ff3-6dd3-4b37-8dcd-383610a86255`::ref 1 aborting False >Thread-40::INFO::2012-12-04 18:47:48,917::sp::542::Storage.StoragePool::(create) spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb poolName=DefaultRest master_sd=8dc966ae-7319-4db8-8fab-33f01a0512b6 domList=['8dc966ae-7319-4db8-8fab-33f01a0512b6'] masterVersion=1 {'LEASETIMESEC': 60, 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3, 'LOCKRENEWALINTERVALSEC': 5} >Thread-40::INFO::2012-12-04 18:47:48,917::fileSD::260::Storage.StorageDomain::(validate) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-40::DEBUG::2012-12-04 18:47:48,919::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=0c95bb213492ef395d7beb8c1df2f8cd4d06d5c3'] >Thread-40::DEBUG::2012-12-04 18:47:48,922::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=0c95bb213492ef395d7beb8c1df2f8cd4d06d5c3'] >Thread-40::DEBUG::2012-12-04 18:47:48,923::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-40::DEBUG::2012-12-04 18:47:48,923::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-40::DEBUG::2012-12-04 18:47:48,923::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=d4e55528681a5ada5d65576cb615330c7c0d2899'] >Thread-40::DEBUG::2012-12-04 18:47:48,926::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-40::INFO::2012-12-04 18:47:48,927::safelease::156::SANLock::(acquireHostId) Acquiring host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 250) >Thread-42::DEBUG::2012-12-04 18:47:48,991::task::588::TaskManager.Task::(_updateState) Task=`6f62414e-b671-4e3b-9cfe-605f75cd29ba`::moving from state init -> state preparing >Thread-42::INFO::2012-12-04 18:47:48,991::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-42::INFO::2012-12-04 18:47:48,991::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-42::DEBUG::2012-12-04 18:47:48,991::task::1172::TaskManager.Task::(prepare) Task=`6f62414e-b671-4e3b-9cfe-605f75cd29ba`::finished: {} >Thread-42::DEBUG::2012-12-04 18:47:48,991::task::588::TaskManager.Task::(_updateState) Task=`6f62414e-b671-4e3b-9cfe-605f75cd29ba`::moving from state preparing -> state finished >Thread-42::DEBUG::2012-12-04 18:47:48,991::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-42::DEBUG::2012-12-04 18:47:48,992::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-42::DEBUG::2012-12-04 18:47:48,992::task::978::TaskManager.Task::(_decref) Task=`6f62414e-b671-4e3b-9cfe-605f75cd29ba`::ref 0 aborting False >Thread-48::DEBUG::2012-12-04 18:47:59,432::task::588::TaskManager.Task::(_updateState) Task=`30297075-740f-4df1-9512-d6d56d7d2dea`::moving from state init -> state preparing >Thread-48::INFO::2012-12-04 18:47:59,432::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-48::INFO::2012-12-04 18:47:59,432::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-48::DEBUG::2012-12-04 18:47:59,432::task::1172::TaskManager.Task::(prepare) Task=`30297075-740f-4df1-9512-d6d56d7d2dea`::finished: {} >Thread-48::DEBUG::2012-12-04 18:47:59,432::task::588::TaskManager.Task::(_updateState) Task=`30297075-740f-4df1-9512-d6d56d7d2dea`::moving from state preparing -> state finished >Thread-48::DEBUG::2012-12-04 18:47:59,432::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-48::DEBUG::2012-12-04 18:47:59,432::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-48::DEBUG::2012-12-04 18:47:59,433::task::978::TaskManager.Task::(_decref) Task=`30297075-740f-4df1-9512-d6d56d7d2dea`::ref 0 aborting False >Thread-54::DEBUG::2012-12-04 18:48:09,782::task::588::TaskManager.Task::(_updateState) Task=`578105a9-c4ab-4205-b58e-05415cd1a465`::moving from state init -> state preparing >Thread-54::INFO::2012-12-04 18:48:09,783::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-54::INFO::2012-12-04 18:48:09,783::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-54::DEBUG::2012-12-04 18:48:09,783::task::1172::TaskManager.Task::(prepare) Task=`578105a9-c4ab-4205-b58e-05415cd1a465`::finished: {} >Thread-54::DEBUG::2012-12-04 18:48:09,783::task::588::TaskManager.Task::(_updateState) Task=`578105a9-c4ab-4205-b58e-05415cd1a465`::moving from state preparing -> state finished >Thread-54::DEBUG::2012-12-04 18:48:09,783::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-54::DEBUG::2012-12-04 18:48:09,783::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-54::DEBUG::2012-12-04 18:48:09,783::task::978::TaskManager.Task::(_decref) Task=`578105a9-c4ab-4205-b58e-05415cd1a465`::ref 0 aborting False >Thread-40::DEBUG::2012-12-04 18:48:09,930::safelease::166::SANLock::(acquireHostId) Host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully acquired (id: 250) >Thread-40::INFO::2012-12-04 18:48:09,930::safelease::188::SANLock::(acquire) Acquiring cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 250) >Thread-40::DEBUG::2012-12-04 18:48:09,966::safelease::213::SANLock::(acquire) Cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully acquired (id: 250) >Thread-40::INFO::2012-12-04 18:48:09,966::sp::721::Storage.StoragePool::(createMaster) setting master domain for spUUID ac5d6452-d8fe-4d69-8d98-d99cd5a494bb on sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-40::DEBUG::2012-12-04 18:48:09,966::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-40::DEBUG::2012-12-04 18:48:09,966::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-40::DEBUG::2012-12-04 18:48:09,970::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=d4e55528681a5ada5d65576cb615330c7c0d2899'] >Thread-40::DEBUG::2012-12-04 18:48:09,970::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-40::DEBUG::2012-12-04 18:48:09,970::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-40::DEBUG::2012-12-04 18:48:09,971::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-40::DEBUG::2012-12-04 18:48:09,971::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-40::DEBUG::2012-12-04 18:48:09,971::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dcc37aec9e7e06a55850eb2b4d67c7771aec71d0'] >Thread-40::DEBUG::2012-12-04 18:48:09,974::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-40::DEBUG::2012-12-04 18:48:09,974::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 verified, version 1 >Thread-40::DEBUG::2012-12-04 18:48:09,974::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-40::DEBUG::2012-12-04 18:48:09,975::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-40::DEBUG::2012-12-04 18:48:09,978::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dcc37aec9e7e06a55850eb2b4d67c7771aec71d0'] >Thread-40::DEBUG::2012-12-04 18:48:09,978::sp::1495::Storage.StoragePool::(updateMonitoringThreads) sp `ac5d6452-d8fe-4d69-8d98-d99cd5a494bb` started monitoring domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` >Thread-40::DEBUG::2012-12-04 18:48:09,979::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-40::DEBUG::2012-12-04 18:48:09,981::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-40::DEBUG::2012-12-04 18:48:09,981::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-40::DEBUG::2012-12-04 18:48:10,000::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-40::DEBUG::2012-12-04 18:48:10,000::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-40::INFO::2012-12-04 18:48:10,004::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 to /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-40::INFO::2012-12-04 18:48:10,004::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 to /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd >Thread-40::INFO::2012-12-04 18:48:10,004::safelease::218::SANLock::(release) Releasing cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 250) >Thread-40::DEBUG::2012-12-04 18:48:10,008::safelease::229::SANLock::(release) Cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully released (id: 250) >Thread-40::INFO::2012-12-04 18:48:10,008::safelease::171::SANLock::(releaseHostId) Releasing host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 250) >Thread-40::DEBUG::2012-12-04 18:48:13,009::safelease::180::SANLock::(releaseHostId) Host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 released successfully (id: None) >Thread-40::INFO::2012-12-04 18:48:13,009::logUtils::39::dispatcher::(wrapper) Run and protect: createStoragePool, Return response: True >Thread-40::DEBUG::2012-12-04 18:48:13,009::task::1172::TaskManager.Task::(prepare) Task=`802d3ff3-6dd3-4b37-8dcd-383610a86255`::finished: True >Thread-40::DEBUG::2012-12-04 18:48:13,009::task::588::TaskManager.Task::(_updateState) Task=`802d3ff3-6dd3-4b37-8dcd-383610a86255`::moving from state preparing -> state finished >Thread-40::DEBUG::2012-12-04 18:48:13,010::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>, 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-40::DEBUG::2012-12-04 18:48:13,010::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-40::DEBUG::2012-12-04 18:48:13,010::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-40::DEBUG::2012-12-04 18:48:13,010::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-40::DEBUG::2012-12-04 18:48:13,010::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-40::DEBUG::2012-12-04 18:48:13,010::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-40::DEBUG::2012-12-04 18:48:13,011::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-40::DEBUG::2012-12-04 18:48:13,011::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-40::DEBUG::2012-12-04 18:48:13,011::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-40::DEBUG::2012-12-04 18:48:13,011::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-40::DEBUG::2012-12-04 18:48:13,011::task::978::TaskManager.Task::(_decref) Task=`802d3ff3-6dd3-4b37-8dcd-383610a86255`::ref 0 aborting False >Thread-59::DEBUG::2012-12-04 18:48:13,170::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-59::DEBUG::2012-12-04 18:48:13,170::task::588::TaskManager.Task::(_updateState) Task=`1bd43981-18b5-4dd0-a487-ced82e0c2eaf`::moving from state init -> state preparing >Thread-59::INFO::2012-12-04 18:48:13,171::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-59::INFO::2012-12-04 18:48:13,171::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-59::DEBUG::2012-12-04 18:48:13,171::task::1172::TaskManager.Task::(prepare) Task=`1bd43981-18b5-4dd0-a487-ced82e0c2eaf`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-59::DEBUG::2012-12-04 18:48:13,171::task::588::TaskManager.Task::(_updateState) Task=`1bd43981-18b5-4dd0-a487-ced82e0c2eaf`::moving from state preparing -> state finished >Thread-59::DEBUG::2012-12-04 18:48:13,171::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-59::DEBUG::2012-12-04 18:48:13,171::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-59::DEBUG::2012-12-04 18:48:13,171::task::978::TaskManager.Task::(_decref) Task=`1bd43981-18b5-4dd0-a487-ced82e0c2eaf`::ref 0 aborting False >Thread-60::DEBUG::2012-12-04 18:48:13,212::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-60::DEBUG::2012-12-04 18:48:13,212::task::588::TaskManager.Task::(_updateState) Task=`fd7689be-3981-43cc-9292-9d544e639d84`::moving from state init -> state preparing >Thread-60::INFO::2012-12-04 18:48:13,212::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-60::DEBUG::2012-12-04 18:48:13,216::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-60::DEBUG::2012-12-04 18:48:13,216::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-60::DEBUG::2012-12-04 18:48:13,216::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-60::DEBUG::2012-12-04 18:48:13,216::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-60::DEBUG::2012-12-04 18:48:13,216::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-60::DEBUG::2012-12-04 18:48:13,217::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-60::INFO::2012-12-04 18:48:13,217::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-60::DEBUG::2012-12-04 18:48:13,217::task::1172::TaskManager.Task::(prepare) Task=`fd7689be-3981-43cc-9292-9d544e639d84`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-60::DEBUG::2012-12-04 18:48:13,217::task::588::TaskManager.Task::(_updateState) Task=`fd7689be-3981-43cc-9292-9d544e639d84`::moving from state preparing -> state finished >Thread-60::DEBUG::2012-12-04 18:48:13,217::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-60::DEBUG::2012-12-04 18:48:13,217::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-60::DEBUG::2012-12-04 18:48:13,217::task::978::TaskManager.Task::(_decref) Task=`fd7689be-3981-43cc-9292-9d544e639d84`::ref 0 aborting False >Thread-61::DEBUG::2012-12-04 18:48:13,306::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-61::DEBUG::2012-12-04 18:48:13,307::task::588::TaskManager.Task::(_updateState) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::moving from state init -> state preparing >Thread-61::INFO::2012-12-04 18:48:13,307::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-61::ERROR::2012-12-04 18:48:13,307::task::853::TaskManager.Task::(_setError) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 861, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 38, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 528, in getSpmStatus > pool = self.getPool(spUUID) > File "/usr/share/vdsm/storage/hsm.py", line 265, in getPool > raise se.StoragePoolUnknown(spUUID) >StoragePoolUnknown: Unknown pool id, pool not connected: ('ac5d6452-d8fe-4d69-8d98-d99cd5a494bb',) >Thread-61::DEBUG::2012-12-04 18:48:13,308::task::872::TaskManager.Task::(_run) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::Task._run: d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b ('ac5d6452-d8fe-4d69-8d98-d99cd5a494bb',) {} failed - stopping task >Thread-61::DEBUG::2012-12-04 18:48:13,309::task::1199::TaskManager.Task::(stop) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::stopping in state preparing (force False) >Thread-61::DEBUG::2012-12-04 18:48:13,309::task::978::TaskManager.Task::(_decref) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::ref 1 aborting True >Thread-61::INFO::2012-12-04 18:48:13,309::task::1157::TaskManager.Task::(prepare) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-61::DEBUG::2012-12-04 18:48:13,309::task::1162::TaskManager.Task::(prepare) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::Prepare: aborted: Unknown pool id, pool not connected >Thread-61::DEBUG::2012-12-04 18:48:13,309::task::978::TaskManager.Task::(_decref) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::ref 0 aborting True >Thread-61::DEBUG::2012-12-04 18:48:13,309::task::913::TaskManager.Task::(_doAbort) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::Task._doAbort: force False >Thread-61::DEBUG::2012-12-04 18:48:13,309::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-61::DEBUG::2012-12-04 18:48:13,310::task::588::TaskManager.Task::(_updateState) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::moving from state preparing -> state aborting >Thread-61::DEBUG::2012-12-04 18:48:13,310::task::537::TaskManager.Task::(__state_aborting) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::_aborting: recover policy none >Thread-61::DEBUG::2012-12-04 18:48:13,310::task::588::TaskManager.Task::(_updateState) Task=`d4d7ee24-2f37-4df2-b5e0-7ac6fefdee8b`::moving from state aborting -> state failed >Thread-61::DEBUG::2012-12-04 18:48:13,310::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-61::DEBUG::2012-12-04 18:48:13,310::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-61::ERROR::2012-12-04 18:48:13,310::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('ac5d6452-d8fe-4d69-8d98-d99cd5a494bb',)", 'code': 309}} >Thread-62::DEBUG::2012-12-04 18:48:13,349::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-62::DEBUG::2012-12-04 18:48:13,350::task::588::TaskManager.Task::(_updateState) Task=`75b6b57c-8717-42b3-917b-0e221a8138a2`::moving from state init -> state preparing >Thread-62::INFO::2012-12-04 18:48:13,350::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', hostID=1, scsiKey='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', msdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', masterVersion=1, options=None) >Thread-62::DEBUG::2012-12-04 18:48:13,350::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`359d7e9c-f903-4548-b3de-8d0beefeed17`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-62::DEBUG::2012-12-04 18:48:13,350::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-62::DEBUG::2012-12-04 18:48:13,351::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-62::DEBUG::2012-12-04 18:48:13,351::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`359d7e9c-f903-4548-b3de-8d0beefeed17`::Granted request >Thread-62::INFO::2012-12-04 18:48:13,351::sp::625::Storage.StoragePool::(connect) Connect host #1 to the storage pool ac5d6452-d8fe-4d69-8d98-d99cd5a494bb with master domain: 8dc966ae-7319-4db8-8fab-33f01a0512b6 (ver = 1) >Thread-62::DEBUG::2012-12-04 18:48:13,351::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:13,352::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:13,352::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:13,352::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:13,352::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:13,352::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:13,352::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-62::DEBUG::2012-12-04 18:48:13,353::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-62::DEBUG::2012-12-04 18:48:13,353::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-62::DEBUG::2012-12-04 18:48:13,353::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-62::DEBUG::2012-12-04 18:48:13,353::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-62::DEBUG::2012-12-04 18:48:13,362::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-62::DEBUG::2012-12-04 18:48:13,363::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-62::DEBUG::2012-12-04 18:48:15,197::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-62::DEBUG::2012-12-04 18:48:15,224::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-62::DEBUG::2012-12-04 18:48:15,224::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,225::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,225::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,225::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,225::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,225::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,226::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-62::DEBUG::2012-12-04 18:48:15,226::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,226::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 8dc966ae-7319-4db8-8fab-33f01a0512b6' (cwd None) >Thread-62::DEBUG::2012-12-04 18:48:15,244::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found\n'; <rc> = 5 >Thread-62::WARNING::2012-12-04 18:48:15,244::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found'] >Thread-62::DEBUG::2012-12-04 18:48:15,244::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,261::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-62::DEBUG::2012-12-04 18:48:15,261::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-62::DEBUG::2012-12-04 18:48:15,272::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dcc37aec9e7e06a55850eb2b4d67c7771aec71d0'] >Thread-62::DEBUG::2012-12-04 18:48:15,283::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-62::WARNING::2012-12-04 18:48:15,283::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS already registered >Thread-62::WARNING::2012-12-04 18:48:15,283::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS already registered >Thread-62::DEBUG::2012-12-04 18:48:15,283::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 verified, version 1 >Thread-62::DEBUG::2012-12-04 18:48:15,283::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-62::DEBUG::2012-12-04 18:48:15,283::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-62::DEBUG::2012-12-04 18:48:15,304::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dcc37aec9e7e06a55850eb2b4d67c7771aec71d0'] >Thread-62::DEBUG::2012-12-04 18:48:15,305::sp::1495::Storage.StoragePool::(updateMonitoringThreads) sp `ac5d6452-d8fe-4d69-8d98-d99cd5a494bb` started monitoring domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` >Thread-62::DEBUG::2012-12-04 18:48:15,306::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-62::DEBUG::2012-12-04 18:48:15,308::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-62::DEBUG::2012-12-04 18:48:15,309::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-62::DEBUG::2012-12-04 18:48:15,327::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-62::DEBUG::2012-12-04 18:48:15,327::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-62::INFO::2012-12-04 18:48:15,330::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 to /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-62::INFO::2012-12-04 18:48:15,330::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 to /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd >Thread-62::INFO::2012-12-04 18:48:15,331::safelease::156::SANLock::(acquireHostId) Acquiring host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 1) >Thread-69::DEBUG::2012-12-04 18:48:20,204::task::588::TaskManager.Task::(_updateState) Task=`6493c25f-c40b-45cb-82af-76f1662fdaef`::moving from state init -> state preparing >Thread-69::INFO::2012-12-04 18:48:20,204::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-69::INFO::2012-12-04 18:48:20,204::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-69::DEBUG::2012-12-04 18:48:20,204::task::1172::TaskManager.Task::(prepare) Task=`6493c25f-c40b-45cb-82af-76f1662fdaef`::finished: {} >Thread-69::DEBUG::2012-12-04 18:48:20,204::task::588::TaskManager.Task::(_updateState) Task=`6493c25f-c40b-45cb-82af-76f1662fdaef`::moving from state preparing -> state finished >Thread-69::DEBUG::2012-12-04 18:48:20,205::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-69::DEBUG::2012-12-04 18:48:20,205::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-69::DEBUG::2012-12-04 18:48:20,205::task::978::TaskManager.Task::(_decref) Task=`6493c25f-c40b-45cb-82af-76f1662fdaef`::ref 0 aborting False >Thread-75::DEBUG::2012-12-04 18:48:30,544::task::588::TaskManager.Task::(_updateState) Task=`d113e832-5436-4749-89aa-ebd95d8b9bf0`::moving from state init -> state preparing >Thread-75::INFO::2012-12-04 18:48:30,545::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-75::INFO::2012-12-04 18:48:30,545::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-75::DEBUG::2012-12-04 18:48:30,545::task::1172::TaskManager.Task::(prepare) Task=`d113e832-5436-4749-89aa-ebd95d8b9bf0`::finished: {} >Thread-75::DEBUG::2012-12-04 18:48:30,545::task::588::TaskManager.Task::(_updateState) Task=`d113e832-5436-4749-89aa-ebd95d8b9bf0`::moving from state preparing -> state finished >Thread-75::DEBUG::2012-12-04 18:48:30,545::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-75::DEBUG::2012-12-04 18:48:30,545::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-75::DEBUG::2012-12-04 18:48:30,545::task::978::TaskManager.Task::(_decref) Task=`d113e832-5436-4749-89aa-ebd95d8b9bf0`::ref 0 aborting False >Thread-62::DEBUG::2012-12-04 18:48:36,334::safelease::166::SANLock::(acquireHostId) Host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully acquired (id: 1) >Thread-62::DEBUG::2012-12-04 18:48:36,334::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-62::DEBUG::2012-12-04 18:48:36,334::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-62::DEBUG::2012-12-04 18:48:36,334::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-62::DEBUG::2012-12-04 18:48:36,334::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-62::INFO::2012-12-04 18:48:36,335::logUtils::39::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-62::DEBUG::2012-12-04 18:48:36,335::task::1172::TaskManager.Task::(prepare) Task=`75b6b57c-8717-42b3-917b-0e221a8138a2`::finished: True >Thread-62::DEBUG::2012-12-04 18:48:36,335::task::588::TaskManager.Task::(_updateState) Task=`75b6b57c-8717-42b3-917b-0e221a8138a2`::moving from state preparing -> state finished >Thread-62::DEBUG::2012-12-04 18:48:36,335::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-62::DEBUG::2012-12-04 18:48:36,335::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-62::DEBUG::2012-12-04 18:48:36,335::task::978::TaskManager.Task::(_decref) Task=`75b6b57c-8717-42b3-917b-0e221a8138a2`::ref 0 aborting False >Thread-79::DEBUG::2012-12-04 18:48:36,378::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-79::DEBUG::2012-12-04 18:48:36,378::task::588::TaskManager.Task::(_updateState) Task=`504b12d7-271d-4256-acbf-d1ce1fa39678`::moving from state init -> state preparing >Thread-79::INFO::2012-12-04 18:48:36,378::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-79::INFO::2012-12-04 18:48:36,378::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-79::DEBUG::2012-12-04 18:48:36,378::task::1172::TaskManager.Task::(prepare) Task=`504b12d7-271d-4256-acbf-d1ce1fa39678`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-79::DEBUG::2012-12-04 18:48:36,379::task::588::TaskManager.Task::(_updateState) Task=`504b12d7-271d-4256-acbf-d1ce1fa39678`::moving from state preparing -> state finished >Thread-79::DEBUG::2012-12-04 18:48:36,379::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-79::DEBUG::2012-12-04 18:48:36,379::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-79::DEBUG::2012-12-04 18:48:36,379::task::978::TaskManager.Task::(_decref) Task=`504b12d7-271d-4256-acbf-d1ce1fa39678`::ref 0 aborting False >Thread-80::DEBUG::2012-12-04 18:48:36,473::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-80::DEBUG::2012-12-04 18:48:36,474::task::588::TaskManager.Task::(_updateState) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::moving from state init -> state preparing >Thread-80::INFO::2012-12-04 18:48:36,474::logUtils::37::dispatcher::(wrapper) Run and protect: spmStart(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', prevID=-1, prevLVER='-1', recoveryMode=None, scsiFencing='false', maxHostID=250, domVersion='3', options=None) >Thread-80::DEBUG::2012-12-04 18:48:36,474::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`463debeb-8d66-46fa-9b4e-ef975d458bdc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-80::DEBUG::2012-12-04 18:48:36,474::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-80::DEBUG::2012-12-04 18:48:36,475::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-80::DEBUG::2012-12-04 18:48:36,475::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`463debeb-8d66-46fa-9b4e-ef975d458bdc`::Granted request >Thread-80::DEBUG::2012-12-04 18:48:36,475::task::817::TaskManager.Task::(resourceAcquired) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (exclusive) >Thread-80::DEBUG::2012-12-04 18:48:36,475::task::978::TaskManager.Task::(_decref) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::ref 1 aborting False >Thread-80::INFO::2012-12-04 18:48:36,475::logUtils::39::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >Thread-80::DEBUG::2012-12-04 18:48:36,476::task::1167::TaskManager.Task::(prepare) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::Prepare: 1 jobs exist, move to acquiring >Thread-80::DEBUG::2012-12-04 18:48:36,476::task::588::TaskManager.Task::(_updateState) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::moving from state preparing -> state acquiring >Thread-80::DEBUG::2012-12-04 18:48:36,476::task::588::TaskManager.Task::(_updateState) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::moving from state acquiring -> state queued >Thread-80::DEBUG::2012-12-04 18:48:36,476::taskManager::48::TaskManager::(_queueTask) queuing task: 6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf >Thread-80::DEBUG::2012-12-04 18:48:36,476::taskManager::54::TaskManager::(_queueTask) task queued: 6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf >Thread-80::DEBUG::2012-12-04 18:48:36,476::task::1169::TaskManager.Task::(prepare) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::returning >Thread-80::DEBUG::2012-12-04 18:48:36,476::task::978::TaskManager.Task::(_decref) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::ref 0 aborting False >Thread-2::DEBUG::2012-12-04 18:48:36,476::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,477::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: 6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf running: <bound method Task.commit of <storage.task.Task instance at 0x7f7f2c271098>> with: None >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,477::task::1181::TaskManager.Task::(commit) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::committing task: 6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,478::task::588::TaskManager.Task::(_updateState) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::moving from state queued -> state running >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,478::task::892::TaskManager.Task::(_runJobs) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f7f2c193e10>> (args: (-1, '-1', 'false', 250, 3) kwargs: {}) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,478::task::317::TaskManager.Task::(run) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f7f2c193e10>> (args: (-1, '-1', 'false', 250, 3) kwargs: {}) callback None >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,478::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,478::misc::1055::SamplingMethod::(__call__) Got in to sampling method >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,482::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dcc37aec9e7e06a55850eb2b4d67c7771aec71d0'] >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,482::misc::1063::SamplingMethod::(__call__) Returning last result >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,485::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dcc37aec9e7e06a55850eb2b4d67c7771aec71d0'] >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::INFO::2012-12-04 18:48:36,485::safelease::188::SANLock::(acquire) Acquiring cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 1) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,523::safelease::213::SANLock::(acquire) Cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully acquired (id: 1) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,523::sp::260::Storage.StoragePool::(startSpm) spm lock acquired successfully >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,527::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dcc37aec9e7e06a55850eb2b4d67c7771aec71d0'] >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,528::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,528::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,528::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRest', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=7598bc33e1b8201ee30ad739f8ec9f2137aaecc8'] >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,533::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,533::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`7a7e6a07-33b8-463f-ac78-163cfbbe9986`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,533::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,534::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,534::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`7a7e6a07-33b8-463f-ac78-163cfbbe9986`::Granted request >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::INFO::2012-12-04 18:48:36,534::sp::407::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,534::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`5d11c9ff-8416-4c68-9e7f-13d86893839f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,534::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'exclusive' >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,535::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'exclusive' (1 active user) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,535::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`5d11c9ff-8416-4c68-9e7f-13d86893839f`::Granted request >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,535::sd::385::Storage.StorageDomain::(upgrade) Trying to upgrade domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` from version 3 to version 3 >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,535::sd::391::Storage.StorageDomain::(upgrade) No need to upgrade domain `8dc966ae-7319-4db8-8fab-33f01a0512b6`, leaving unchanged >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,535::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,536::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,536::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,536::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,536::sp::411::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,536::sp::418::Storage.StoragePool::(_upgradePool) Registering with state change event >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,536::sp::420::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,536::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,537::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,537::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,537::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::WARNING::2012-12-04 18:48:36,541::sd::564::Storage.StorageDomain::(createMasterTree) vms dir not found, creating (/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6/master/vms) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::WARNING::2012-12-04 18:48:36,542::sd::569::Storage.StorageDomain::(createMasterTree) tasks dir not found, creating (/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6/master/tasks) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,545::sp::309::Storage.StoragePool::(startSpm) ended. >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,545::task::588::TaskManager.Task::(_updateState) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::moving from state running -> state finished >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,545::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,545::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,545::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,545::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,545::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,546::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,546::task::900::TaskManager.Task::(_runJobs) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::Task.run: exit - success: result >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,546::task::978::TaskManager.Task::(_decref) Task=`6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf`::ref 0 aborting False >6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf::DEBUG::2012-12-04 18:48:36,546::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-82::DEBUG::2012-12-04 18:48:37,514::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-82::DEBUG::2012-12-04 18:48:37,514::task::588::TaskManager.Task::(_updateState) Task=`ef8bc90c-1f8a-4ae5-a63b-a75be3e0db56`::moving from state init -> state preparing >Thread-82::INFO::2012-12-04 18:48:37,514::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf', spUUID=None, options=None) >Thread-82::DEBUG::2012-12-04 18:48:37,514::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf >Thread-82::DEBUG::2012-12-04 18:48:37,514::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf'} >Thread-82::INFO::2012-12-04 18:48:37,515::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf'}} >Thread-82::DEBUG::2012-12-04 18:48:37,515::task::1172::TaskManager.Task::(prepare) Task=`ef8bc90c-1f8a-4ae5-a63b-a75be3e0db56`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf'}} >Thread-82::DEBUG::2012-12-04 18:48:37,515::task::588::TaskManager.Task::(_updateState) Task=`ef8bc90c-1f8a-4ae5-a63b-a75be3e0db56`::moving from state preparing -> state finished >Thread-82::DEBUG::2012-12-04 18:48:37,515::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-82::DEBUG::2012-12-04 18:48:37,515::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-82::DEBUG::2012-12-04 18:48:37,515::task::978::TaskManager.Task::(_decref) Task=`ef8bc90c-1f8a-4ae5-a63b-a75be3e0db56`::ref 0 aborting False >Thread-83::DEBUG::2012-12-04 18:48:37,554::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-83::DEBUG::2012-12-04 18:48:37,554::task::588::TaskManager.Task::(_updateState) Task=`4dcfe8d1-93b7-4f4c-850f-4f78fe2cec84`::moving from state init -> state preparing >Thread-83::INFO::2012-12-04 18:48:37,554::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-83::INFO::2012-12-04 18:48:37,555::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-83::DEBUG::2012-12-04 18:48:37,555::task::1172::TaskManager.Task::(prepare) Task=`4dcfe8d1-93b7-4f4c-850f-4f78fe2cec84`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-83::DEBUG::2012-12-04 18:48:37,555::task::588::TaskManager.Task::(_updateState) Task=`4dcfe8d1-93b7-4f4c-850f-4f78fe2cec84`::moving from state preparing -> state finished >Thread-83::DEBUG::2012-12-04 18:48:37,555::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-83::DEBUG::2012-12-04 18:48:37,555::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-83::DEBUG::2012-12-04 18:48:37,555::task::978::TaskManager.Task::(_decref) Task=`4dcfe8d1-93b7-4f4c-850f-4f78fe2cec84`::ref 0 aborting False >Thread-84::DEBUG::2012-12-04 18:48:37,589::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-84::DEBUG::2012-12-04 18:48:37,589::task::588::TaskManager.Task::(_updateState) Task=`0470f5bc-1c95-4dee-ac2b-b14f71a415fa`::moving from state init -> state preparing >Thread-84::INFO::2012-12-04 18:48:37,589::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf', spUUID=None, options=None) >Thread-84::DEBUG::2012-12-04 18:48:37,589::taskManager::161::TaskManager::(clearTask) Entry. taskID: 6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf >Thread-84::DEBUG::2012-12-04 18:48:37,590::taskManager::166::TaskManager::(clearTask) Return. >Thread-84::INFO::2012-12-04 18:48:37,590::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-84::DEBUG::2012-12-04 18:48:37,590::task::1172::TaskManager.Task::(prepare) Task=`0470f5bc-1c95-4dee-ac2b-b14f71a415fa`::finished: None >Thread-84::DEBUG::2012-12-04 18:48:37,590::task::588::TaskManager.Task::(_updateState) Task=`0470f5bc-1c95-4dee-ac2b-b14f71a415fa`::moving from state preparing -> state finished >Thread-84::DEBUG::2012-12-04 18:48:37,590::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-84::DEBUG::2012-12-04 18:48:37,590::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-84::DEBUG::2012-12-04 18:48:37,590::task::978::TaskManager.Task::(_decref) Task=`0470f5bc-1c95-4dee-ac2b-b14f71a415fa`::ref 0 aborting False >Thread-85::DEBUG::2012-12-04 18:48:37,676::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-85::DEBUG::2012-12-04 18:48:37,676::task::588::TaskManager.Task::(_updateState) Task=`6ca9fd8b-46ee-416a-9238-3118f29d13cf`::moving from state init -> state preparing >Thread-85::INFO::2012-12-04 18:48:37,676::logUtils::37::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-85::DEBUG::2012-12-04 18:48:37,676::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`f9759d52-2cfe-4e51-b999-51dbcd27c043`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-85::DEBUG::2012-12-04 18:48:37,677::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-85::DEBUG::2012-12-04 18:48:37,677::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-85::DEBUG::2012-12-04 18:48:37,677::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`f9759d52-2cfe-4e51-b999-51dbcd27c043`::Granted request >Thread-85::DEBUG::2012-12-04 18:48:37,677::task::817::TaskManager.Task::(resourceAcquired) Task=`6ca9fd8b-46ee-416a-9238-3118f29d13cf`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (exclusive) >Thread-85::DEBUG::2012-12-04 18:48:37,677::task::978::TaskManager.Task::(_decref) Task=`6ca9fd8b-46ee-416a-9238-3118f29d13cf`::ref 1 aborting False >Thread-85::DEBUG::2012-12-04 18:48:37,678::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`59e72ef4-e44c-4354-b3f9-cecb7460f1eb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-85::DEBUG::2012-12-04 18:48:37,678::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'exclusive' >Thread-85::DEBUG::2012-12-04 18:48:37,678::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'exclusive' (1 active user) >Thread-85::DEBUG::2012-12-04 18:48:37,678::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`59e72ef4-e44c-4354-b3f9-cecb7460f1eb`::Granted request >Thread-85::DEBUG::2012-12-04 18:48:37,678::task::817::TaskManager.Task::(resourceAcquired) Task=`6ca9fd8b-46ee-416a-9238-3118f29d13cf`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (exclusive) >Thread-85::DEBUG::2012-12-04 18:48:37,679::task::978::TaskManager.Task::(_decref) Task=`6ca9fd8b-46ee-416a-9238-3118f29d13cf`::ref 1 aborting False >Thread-85::INFO::2012-12-04 18:48:37,679::sp::986::Storage.StoragePool::(activateSD) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb >Thread-85::INFO::2012-12-04 18:48:37,679::logUtils::39::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-85::DEBUG::2012-12-04 18:48:37,679::task::1172::TaskManager.Task::(prepare) Task=`6ca9fd8b-46ee-416a-9238-3118f29d13cf`::finished: None >Thread-85::DEBUG::2012-12-04 18:48:37,679::task::588::TaskManager.Task::(_updateState) Task=`6ca9fd8b-46ee-416a-9238-3118f29d13cf`::moving from state preparing -> state finished >Thread-85::DEBUG::2012-12-04 18:48:37,679::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>, 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-85::DEBUG::2012-12-04 18:48:37,679::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2012-12-04 18:48:37,680::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-85::DEBUG::2012-12-04 18:48:37,680::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-85::DEBUG::2012-12-04 18:48:37,680::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-85::DEBUG::2012-12-04 18:48:37,680::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-85::DEBUG::2012-12-04 18:48:37,680::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-85::DEBUG::2012-12-04 18:48:37,680::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-85::DEBUG::2012-12-04 18:48:37,681::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-85::DEBUG::2012-12-04 18:48:37,681::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-85::DEBUG::2012-12-04 18:48:37,681::task::978::TaskManager.Task::(_decref) Task=`6ca9fd8b-46ee-416a-9238-3118f29d13cf`::ref 0 aborting False >Thread-86::DEBUG::2012-12-04 18:48:37,712::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-86::DEBUG::2012-12-04 18:48:37,712::task::588::TaskManager.Task::(_updateState) Task=`4f4addb8-b9d8-4812-8d48-920cde436240`::moving from state init -> state preparing >Thread-86::INFO::2012-12-04 18:48:37,712::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', hostID=1, scsiKey='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', msdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', masterVersion=1, options=None) >Thread-86::DEBUG::2012-12-04 18:48:37,713::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`881f8693-a585-4439-9e7d-c9f71bfb018e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-86::DEBUG::2012-12-04 18:48:37,713::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-86::DEBUG::2012-12-04 18:48:37,713::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-86::DEBUG::2012-12-04 18:48:37,713::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`881f8693-a585-4439-9e7d-c9f71bfb018e`::Granted request >Thread-86::DEBUG::2012-12-04 18:48:37,714::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 verified, version 1 >Thread-86::DEBUG::2012-12-04 18:48:37,714::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-86::DEBUG::2012-12-04 18:48:37,714::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-86::DEBUG::2012-12-04 18:48:37,714::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-86::DEBUG::2012-12-04 18:48:37,714::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-86::INFO::2012-12-04 18:48:37,714::logUtils::39::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: None >Thread-86::DEBUG::2012-12-04 18:48:37,715::task::1172::TaskManager.Task::(prepare) Task=`4f4addb8-b9d8-4812-8d48-920cde436240`::finished: None >Thread-86::DEBUG::2012-12-04 18:48:37,715::task::588::TaskManager.Task::(_updateState) Task=`4f4addb8-b9d8-4812-8d48-920cde436240`::moving from state preparing -> state finished >Thread-86::DEBUG::2012-12-04 18:48:37,715::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2012-12-04 18:48:37,715::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2012-12-04 18:48:37,715::task::978::TaskManager.Task::(_decref) Task=`4f4addb8-b9d8-4812-8d48-920cde436240`::ref 0 aborting False >Thread-88::DEBUG::2012-12-04 18:48:39,163::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-88::DEBUG::2012-12-04 18:48:39,163::task::588::TaskManager.Task::(_updateState) Task=`f02a07ab-bff2-43e9-a90f-72f38cf24cb1`::moving from state init -> state preparing >Thread-88::INFO::2012-12-04 18:48:39,163::logUtils::37::dispatcher::(wrapper) Run and protect: setStorageDomainDescription(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', description='DataDomainRestUpd', options=None) >Thread-88::DEBUG::2012-12-04 18:48:39,163::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`5e3b69ad-805c-476d-b4a5-1586c29cf164`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-88::DEBUG::2012-12-04 18:48:39,163::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'exclusive' >Thread-88::DEBUG::2012-12-04 18:48:39,164::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'exclusive' (1 active user) >Thread-88::DEBUG::2012-12-04 18:48:39,164::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`5e3b69ad-805c-476d-b4a5-1586c29cf164`::Granted request >Thread-88::DEBUG::2012-12-04 18:48:39,164::task::817::TaskManager.Task::(resourceAcquired) Task=`f02a07ab-bff2-43e9-a90f-72f38cf24cb1`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (exclusive) >Thread-88::DEBUG::2012-12-04 18:48:39,164::task::978::TaskManager.Task::(_decref) Task=`f02a07ab-bff2-43e9-a90f-72f38cf24cb1`::ref 1 aborting False >Thread-88::INFO::2012-12-04 18:48:39,164::sd::621::Storage.StorageDomain::(setDescription) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 descr=DataDomainRestUpd >Thread-88::DEBUG::2012-12-04 18:48:39,164::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-88::DEBUG::2012-12-04 18:48:39,165::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-88::DEBUG::2012-12-04 18:48:39,165::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=49966f75e3c1291add4e8ecc44a5ac665a2d7fe0'] >Thread-88::DEBUG::2012-12-04 18:48:39,170::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-88::INFO::2012-12-04 18:48:39,171::logUtils::39::dispatcher::(wrapper) Run and protect: setStorageDomainDescription, Return response: None >Thread-88::DEBUG::2012-12-04 18:48:39,171::task::1172::TaskManager.Task::(prepare) Task=`f02a07ab-bff2-43e9-a90f-72f38cf24cb1`::finished: None >Thread-88::DEBUG::2012-12-04 18:48:39,171::task::588::TaskManager.Task::(_updateState) Task=`f02a07ab-bff2-43e9-a90f-72f38cf24cb1`::moving from state preparing -> state finished >Thread-88::DEBUG::2012-12-04 18:48:39,171::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >Thread-88::DEBUG::2012-12-04 18:48:39,171::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-88::DEBUG::2012-12-04 18:48:39,171::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-88::DEBUG::2012-12-04 18:48:39,171::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-88::DEBUG::2012-12-04 18:48:39,172::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-88::DEBUG::2012-12-04 18:48:39,172::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-88::DEBUG::2012-12-04 18:48:39,172::task::978::TaskManager.Task::(_decref) Task=`f02a07ab-bff2-43e9-a90f-72f38cf24cb1`::ref 0 aborting False >Thread-89::DEBUG::2012-12-04 18:48:41,133::task::588::TaskManager.Task::(_updateState) Task=`4fe940a7-4045-4903-9d3b-72b6d2d4f6a5`::moving from state init -> state preparing >Thread-89::INFO::2012-12-04 18:48:41,134::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::WARNING::2012-12-04 18:48:41,134::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:8dc966ae-7319-4db8-8fab-33f01a0512b6 {'finish': 1354639715.338887, 'mdasize': 0, 'result': {'delay': '0.00241184234619', 'lastCheck': 1354639715.338887, 'code': 0, 'valid': True}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': True}, 'diskfree': '844027133952', 'mdafree': 0, 'disktotal': '844029231104', 'mdathreshold': True} >Thread-89::INFO::2012-12-04 18:48:41,134::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00241184234619', 'lastCheck': 1354639715.338887, 'code': 355, 'valid': True}} >Thread-89::DEBUG::2012-12-04 18:48:41,134::task::1172::TaskManager.Task::(prepare) Task=`4fe940a7-4045-4903-9d3b-72b6d2d4f6a5`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00241184234619', 'lastCheck': 1354639715.338887, 'code': 355, 'valid': True}} >Thread-89::DEBUG::2012-12-04 18:48:41,134::task::588::TaskManager.Task::(_updateState) Task=`4fe940a7-4045-4903-9d3b-72b6d2d4f6a5`::moving from state preparing -> state finished >Thread-89::DEBUG::2012-12-04 18:48:41,134::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2012-12-04 18:48:41,135::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2012-12-04 18:48:41,135::task::978::TaskManager.Task::(_decref) Task=`4fe940a7-4045-4903-9d3b-72b6d2d4f6a5`::ref 0 aborting False >Thread-93::DEBUG::2012-12-04 18:48:45,909::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-93::DEBUG::2012-12-04 18:48:45,909::task::588::TaskManager.Task::(_updateState) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::moving from state init -> state preparing >Thread-93::INFO::2012-12-04 18:48:45,909::logUtils::37::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', imgUUID='54804768-5a65-49d2-98b6-1a95b7799d8b', size='2147483648', volFormat=4, preallocate=2, diskType=2, volUUID='bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', desc='', srcImgUUID='00000000-0000-0000-0000-000000000000', srcVolUUID='00000000-0000-0000-0000-000000000000') >Thread-93::INFO::2012-12-04 18:48:45,909::fileSD::260::Storage.StorageDomain::(validate) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-93::DEBUG::2012-12-04 18:48:46,177::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=49966f75e3c1291add4e8ecc44a5ac665a2d7fe0'] >Thread-93::DEBUG::2012-12-04 18:48:46,178::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`04791a00-8bd4-4099-ad4e-ca1d8eb5ec9c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-93::DEBUG::2012-12-04 18:48:46,178::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-93::DEBUG::2012-12-04 18:48:46,179::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-93::DEBUG::2012-12-04 18:48:46,179::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`04791a00-8bd4-4099-ad4e-ca1d8eb5ec9c`::Granted request >Thread-93::DEBUG::2012-12-04 18:48:46,179::task::817::TaskManager.Task::(resourceAcquired) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-93::DEBUG::2012-12-04 18:48:46,179::task::978::TaskManager.Task::(_decref) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::ref 1 aborting False >Thread-93::DEBUG::2012-12-04 18:48:46,233::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >Thread-93::DEBUG::2012-12-04 18:48:46,455::taskManager::66::TaskManager::(scheduleJob) scheduled job createVolume for task d0cfa4da-1b3a-47be-9155-837c4b82b5aa >Thread-93::INFO::2012-12-04 18:48:46,455::logUtils::39::dispatcher::(wrapper) Run and protect: createVolume, Return response: None >Thread-93::DEBUG::2012-12-04 18:48:46,455::task::1167::TaskManager.Task::(prepare) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::Prepare: 1 jobs exist, move to acquiring >Thread-93::DEBUG::2012-12-04 18:48:46,455::task::588::TaskManager.Task::(_updateState) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::moving from state preparing -> state acquiring >Thread-93::DEBUG::2012-12-04 18:48:46,460::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >Thread-93::DEBUG::2012-12-04 18:48:46,596::task::588::TaskManager.Task::(_updateState) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::moving from state acquiring -> state queued >Thread-93::DEBUG::2012-12-04 18:48:46,600::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >Thread-93::DEBUG::2012-12-04 18:48:46,685::taskManager::48::TaskManager::(_queueTask) queuing task: d0cfa4da-1b3a-47be-9155-837c4b82b5aa >Thread-93::DEBUG::2012-12-04 18:48:46,685::taskManager::54::TaskManager::(_queueTask) task queued: d0cfa4da-1b3a-47be-9155-837c4b82b5aa >Thread-8::DEBUG::2012-12-04 18:48:46,686::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-93::DEBUG::2012-12-04 18:48:46,686::task::1169::TaskManager.Task::(prepare) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::returning >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,686::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: d0cfa4da-1b3a-47be-9155-837c4b82b5aa running: <bound method Task.commit of <storage.task.Task instance at 0x7f7f2c278830>> with: None >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,686::task::1181::TaskManager.Task::(commit) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::committing task: d0cfa4da-1b3a-47be-9155-837c4b82b5aa >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,686::task::588::TaskManager.Task::(_updateState) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::moving from state queued -> state running >Thread-93::DEBUG::2012-12-04 18:48:46,687::task::978::TaskManager.Task::(_decref) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::ref 0 aborting False >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,688::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,738::task::892::TaskManager.Task::(_runJobs) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::Task.run: running job 0: createVolume: <bound method StoragePool.createVolume of <storage.sp.StoragePool object at 0x7f7f2c193e10>> (args: ('8dc966ae-7319-4db8-8fab-33f01a0512b6', '54804768-5a65-49d2-98b6-1a95b7799d8b', 4194304, 4, 2, 2, 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', '', '00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-000000000000') kwargs: {}) >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,738::task::317::TaskManager.Task::(run) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::Job.run: running createVolume: <bound method StoragePool.createVolume of <storage.sp.StoragePool object at 0x7f7f2c193e10>> (args: ('8dc966ae-7319-4db8-8fab-33f01a0512b6', '54804768-5a65-49d2-98b6-1a95b7799d8b', 4194304, 4, 2, 2, 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', '', '00000000-0000-0000-0000-000000000000', '00000000-0000-0000-0000-000000000000') kwargs: {}) callback None >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,739::resourceManager::175::ResourceManager.Request::(__init__) ResName=`8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b`ReqID=`40578fb6-5c37-42ab-b767-cc42121e8118`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,739::resourceManager::486::ResourceManager::(registerResource) Trying to register resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' for lock type 'exclusive' >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,741::resourceFactories::119::Storage.ResourcesFactories::(__getResourceCandidatesList) Image 54804768-5a65-49d2-98b6-1a95b7799d8b does not exist in domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,741::resourceManager::528::ResourceManager::(registerResource) Resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' is free. Now locking as 'exclusive' (1 active user) >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,741::resourceManager::212::ResourceManager.Request::(grant) ResName=`8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b`ReqID=`40578fb6-5c37-42ab-b767-cc42121e8118`::Granted request >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::INFO::2012-12-04 18:48:46,742::image::94::Storage.Image::(create) Create placeholder /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/54804768-5a65-49d2-98b6-1a95b7799d8b for image's volumes >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,743::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,808::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,889::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::INFO::2012-12-04 18:48:46,980::fileVolume::171::Storage.Volume::(create) fileVolume: create: volUUID bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d srcImg 00000000-0000-0000-0000-000000000000 srvVol 00000000-0000-0000-0000-000000000000 >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::INFO::2012-12-04 18:48:46,980::fileVolume::174::Storage.Volume::(create) Request to create COW volume /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d with size = 4194304 sectors >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:46,981::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/qemu-img create -f qcow2 /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d 2097152K' (cwd None) >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,069::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,086::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,426::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,532::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,532::resourceManager::553::ResourceManager::(releaseResource) Released resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' (0 active users) >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,532::resourceManager::558::ResourceManager::(releaseResource) Resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' is free, finding out if anyone is waiting for it. >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,532::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b', Clearing records. >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,532::task::588::TaskManager.Task::(_updateState) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::moving from state running -> state finished >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,537::task::740::TaskManager.Task::(_save) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/d0cfa4da-1b3a-47be-9155-837c4b82b5aa.temp >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,633::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,634::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,634::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,634::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,634::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,634::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,634::task::900::TaskManager.Task::(_runJobs) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::Task.run: exit - success: result {'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d'} >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,635::task::978::TaskManager.Task::(_decref) Task=`d0cfa4da-1b3a-47be-9155-837c4b82b5aa`::ref 0 aborting False >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:47,635::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-95::DEBUG::2012-12-04 18:48:47,740::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-95::DEBUG::2012-12-04 18:48:47,740::task::588::TaskManager.Task::(_updateState) Task=`a7b3af3e-83a9-4b8a-a438-1c0e0fdeb8e0`::moving from state init -> state preparing >Thread-95::INFO::2012-12-04 18:48:47,740::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-95::INFO::2012-12-04 18:48:47,740::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-95::DEBUG::2012-12-04 18:48:47,741::task::1172::TaskManager.Task::(prepare) Task=`a7b3af3e-83a9-4b8a-a438-1c0e0fdeb8e0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-95::DEBUG::2012-12-04 18:48:47,741::task::588::TaskManager.Task::(_updateState) Task=`a7b3af3e-83a9-4b8a-a438-1c0e0fdeb8e0`::moving from state preparing -> state finished >Thread-95::DEBUG::2012-12-04 18:48:47,741::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2012-12-04 18:48:47,741::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2012-12-04 18:48:47,741::task::978::TaskManager.Task::(_decref) Task=`a7b3af3e-83a9-4b8a-a438-1c0e0fdeb8e0`::ref 0 aborting False >Thread-96::DEBUG::2012-12-04 18:48:47,789::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-96::DEBUG::2012-12-04 18:48:47,789::task::588::TaskManager.Task::(_updateState) Task=`f38217e5-a9ab-4c2e-bedd-32ca73000573`::moving from state init -> state preparing >Thread-96::INFO::2012-12-04 18:48:47,789::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-96::DEBUG::2012-12-04 18:48:47,790::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`b597efb7-3233-4fa1-9cf3-a1ae37bf98b0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-96::DEBUG::2012-12-04 18:48:47,790::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-96::DEBUG::2012-12-04 18:48:47,790::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-96::DEBUG::2012-12-04 18:48:47,790::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`b597efb7-3233-4fa1-9cf3-a1ae37bf98b0`::Granted request >Thread-96::DEBUG::2012-12-04 18:48:47,791::task::817::TaskManager.Task::(resourceAcquired) Task=`f38217e5-a9ab-4c2e-bedd-32ca73000573`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-96::DEBUG::2012-12-04 18:48:47,791::task::978::TaskManager.Task::(_decref) Task=`f38217e5-a9ab-4c2e-bedd-32ca73000573`::ref 1 aborting False >Thread-96::INFO::2012-12-04 18:48:47,795::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844190711808', 'alerts': [], 'disktotal': '844192808960'}}} >Thread-96::DEBUG::2012-12-04 18:48:47,795::task::1172::TaskManager.Task::(prepare) Task=`f38217e5-a9ab-4c2e-bedd-32ca73000573`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844190711808', 'alerts': [], 'disktotal': '844192808960'}}} >Thread-96::DEBUG::2012-12-04 18:48:47,795::task::588::TaskManager.Task::(_updateState) Task=`f38217e5-a9ab-4c2e-bedd-32ca73000573`::moving from state preparing -> state finished >Thread-96::DEBUG::2012-12-04 18:48:47,795::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-96::DEBUG::2012-12-04 18:48:47,795::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-96::DEBUG::2012-12-04 18:48:47,795::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-96::DEBUG::2012-12-04 18:48:47,796::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-96::DEBUG::2012-12-04 18:48:47,796::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-96::DEBUG::2012-12-04 18:48:47,796::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-96::DEBUG::2012-12-04 18:48:47,796::task::978::TaskManager.Task::(_decref) Task=`f38217e5-a9ab-4c2e-bedd-32ca73000573`::ref 0 aborting False >Thread-98::DEBUG::2012-12-04 18:48:51,548::task::588::TaskManager.Task::(_updateState) Task=`a40ab62d-97bb-4064-acf4-5de3f4ab72b9`::moving from state init -> state preparing >Thread-98::INFO::2012-12-04 18:48:51,548::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-98::INFO::2012-12-04 18:48:51,548::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0515310764313', 'lastCheck': 1354639726.194362, 'code': 0, 'valid': True}} >Thread-98::DEBUG::2012-12-04 18:48:51,548::task::1172::TaskManager.Task::(prepare) Task=`a40ab62d-97bb-4064-acf4-5de3f4ab72b9`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0515310764313', 'lastCheck': 1354639726.194362, 'code': 0, 'valid': True}} >Thread-98::DEBUG::2012-12-04 18:48:51,549::task::588::TaskManager.Task::(_updateState) Task=`a40ab62d-97bb-4064-acf4-5de3f4ab72b9`::moving from state preparing -> state finished >Thread-98::DEBUG::2012-12-04 18:48:51,549::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-98::DEBUG::2012-12-04 18:48:51,549::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-98::DEBUG::2012-12-04 18:48:51,549::task::978::TaskManager.Task::(_decref) Task=`a40ab62d-97bb-4064-acf4-5de3f4ab72b9`::ref 0 aborting False >Thread-101::DEBUG::2012-12-04 18:48:54,249::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-101::DEBUG::2012-12-04 18:48:54,249::task::588::TaskManager.Task::(_updateState) Task=`8c8523de-b92d-44ab-930c-373d1c0f30e2`::moving from state init -> state preparing >Thread-101::INFO::2012-12-04 18:48:54,249::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-101::DEBUG::2012-12-04 18:48:54,249::taskManager::103::TaskManager::(getAllTasksStatuses) Entry. >Thread-101::DEBUG::2012-12-04 18:48:54,249::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: d0cfa4da-1b3a-47be-9155-837c4b82b5aa >Thread-101::DEBUG::2012-12-04 18:48:54,249::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'd0cfa4da-1b3a-47be-9155-837c4b82b5aa'} >Thread-101::DEBUG::2012-12-04 18:48:54,250::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {'d0cfa4da-1b3a-47be-9155-837c4b82b5aa': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'd0cfa4da-1b3a-47be-9155-837c4b82b5aa'}} >Thread-101::INFO::2012-12-04 18:48:54,250::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'d0cfa4da-1b3a-47be-9155-837c4b82b5aa': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'd0cfa4da-1b3a-47be-9155-837c4b82b5aa'}}} >Thread-101::DEBUG::2012-12-04 18:48:54,250::task::1172::TaskManager.Task::(prepare) Task=`8c8523de-b92d-44ab-930c-373d1c0f30e2`::finished: {'allTasksStatus': {'d0cfa4da-1b3a-47be-9155-837c4b82b5aa': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'd0cfa4da-1b3a-47be-9155-837c4b82b5aa'}}} >Thread-101::DEBUG::2012-12-04 18:48:54,250::task::588::TaskManager.Task::(_updateState) Task=`8c8523de-b92d-44ab-930c-373d1c0f30e2`::moving from state preparing -> state finished >Thread-101::DEBUG::2012-12-04 18:48:54,250::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-101::DEBUG::2012-12-04 18:48:54,250::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-101::DEBUG::2012-12-04 18:48:54,250::task::978::TaskManager.Task::(_decref) Task=`8c8523de-b92d-44ab-930c-373d1c0f30e2`::ref 0 aborting False >Thread-102::DEBUG::2012-12-04 18:48:54,383::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-102::DEBUG::2012-12-04 18:48:54,384::task::588::TaskManager.Task::(_updateState) Task=`318222f0-6214-4ee6-9ecc-be2489f9eb7f`::moving from state init -> state preparing >Thread-102::INFO::2012-12-04 18:48:54,384::logUtils::37::dispatcher::(wrapper) Run and protect: getVolumeInfo(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', imgUUID='54804768-5a65-49d2-98b6-1a95b7799d8b', volUUID='bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', options=None) >Thread-102::DEBUG::2012-12-04 18:48:54,384::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`b13960b8-04db-4d02-a708-e2892936a187`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-102::DEBUG::2012-12-04 18:48:54,384::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-102::DEBUG::2012-12-04 18:48:54,384::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-102::DEBUG::2012-12-04 18:48:54,385::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`b13960b8-04db-4d02-a708-e2892936a187`::Granted request >Thread-102::DEBUG::2012-12-04 18:48:54,385::task::817::TaskManager.Task::(resourceAcquired) Task=`318222f0-6214-4ee6-9ecc-be2489f9eb7f`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-102::DEBUG::2012-12-04 18:48:54,385::task::978::TaskManager.Task::(_decref) Task=`318222f0-6214-4ee6-9ecc-be2489f9eb7f`::ref 1 aborting False >Thread-102::DEBUG::2012-12-04 18:48:54,386::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-102::INFO::2012-12-04 18:48:54,389::volume::604::Storage.Volume::(getInfo) Info request: sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 imgUUID=54804768-5a65-49d2-98b6-1a95b7799d8b volUUID = bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-102::DEBUG::2012-12-04 18:48:54,402::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-102::DEBUG::2012-12-04 18:48:54,405::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-102::INFO::2012-12-04 18:48:54,409::volume::632::Storage.Volume::(getInfo) 8dc966ae-7319-4db8-8fab-33f01a0512b6/54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d info is {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''} >Thread-102::INFO::2012-12-04 18:48:54,410::logUtils::39::dispatcher::(wrapper) Run and protect: getVolumeInfo, Return response: {'info': {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''}} >Thread-102::DEBUG::2012-12-04 18:48:54,410::task::1172::TaskManager.Task::(prepare) Task=`318222f0-6214-4ee6-9ecc-be2489f9eb7f`::finished: {'info': {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''}} >Thread-102::DEBUG::2012-12-04 18:48:54,410::task::588::TaskManager.Task::(_updateState) Task=`318222f0-6214-4ee6-9ecc-be2489f9eb7f`::moving from state preparing -> state finished >Thread-102::DEBUG::2012-12-04 18:48:54,410::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >Thread-102::DEBUG::2012-12-04 18:48:54,410::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2012-12-04 18:48:54,410::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-102::DEBUG::2012-12-04 18:48:54,411::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-102::DEBUG::2012-12-04 18:48:54,411::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-102::DEBUG::2012-12-04 18:48:54,411::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-102::DEBUG::2012-12-04 18:48:54,411::task::978::TaskManager.Task::(_decref) Task=`318222f0-6214-4ee6-9ecc-be2489f9eb7f`::ref 0 aborting False >Thread-103::DEBUG::2012-12-04 18:48:54,619::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-103::DEBUG::2012-12-04 18:48:54,620::task::588::TaskManager.Task::(_updateState) Task=`cac8c011-662b-4a04-9bdd-86218b6ff07d`::moving from state init -> state preparing >Thread-103::INFO::2012-12-04 18:48:54,620::logUtils::37::dispatcher::(wrapper) Run and protect: updateVM(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', vmList=[{'imglist': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'ovf': '<?xml version="1.0" encoding="UTF-8"?><ovf:Envelope xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1/" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ovf:version="3.2.0.0"><References><File ovf:href="54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d" ovf:id="bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d" ovf:size="2147483648" ovf:description="Active VM"></File></References><Section xsi:type="ovf:NetworkSection_Type"><Info>List of networks</Info><Network ovf:name="Network 1"></Network></Section><Section xsi:type="ovf:DiskSection_Type"><Info>List of Virtual Disks</Info><Disk ovf:diskId="bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d" ovf:size="2" ovf:actual_size="0" ovf:vm_snapshot_id="e35e0fa9-af7b-444b-9d43-997e34d4f2c7" ovf:parentRef="" ovf:fileRef="54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d" ovf:format="http://www.gnome.org/~markmc/qcow-image-format.html" ovf:volume-format="COW" ovf:volume-type="Sparse" ovf:disk-interface="IDE" ovf:boot="false" ovf:disk-alias="restvm_templates_Disk1" ovf:wipe-after-delete="false"></Disk></Section><Content ovf:id="out" xsi:type="ovf:VirtualSystem_Type"><Description></Description><Domain></Domain><CreationDate>2012/12/04 16:48:41</CreationDate><ExportDate>2012/12/04 16:48:54</ExportDate><IsAutoSuspend>false</IsAutoSuspend><IsSmartcardEnabled>false</IsSmartcardEnabled><TimeZone></TimeZone><default_boot_sequence>0</default_boot_sequence><VmType>0</VmType><Name>restvm_templates</Name><TemplateId>00000000-0000-0000-0000-000000000000</TemplateId><TemplateName>Blank</TemplateName><IsInitilized>false</IsInitilized><IsStateless>false</IsStateless><Origin>3</Origin><quota_id>00000000-0000-0000-0000-000000000000</quota_id><DefaultDisplayType>1</DefaultDisplayType><MinAllocatedMem>1024</MinAllocatedMem><Section ovf:id="6ef07ec2-d19d-47f2-974f-f18bd8760545" ovf:required="false" xsi:type="ovf:OperatingSystemSection_Type"><Info>Guest Operating System</Info><Description>Unassigned</Description></Section><Section xsi:type="ovf:VirtualHardwareSection_Type"><Info>1 CPU, 1024 Memeory</Info><System><vssd:VirtualSystemType>ENGINE 3.2.0.0</vssd:VirtualSystemType></System><Item><rasd:Caption>1 virtual cpu</rasd:Caption><rasd:Description>Number of virtual CPU</rasd:Description><rasd:InstanceId>1</rasd:InstanceId><rasd:ResourceType>3</rasd:ResourceType><rasd:num_of_sockets>1</rasd:num_of_sockets><rasd:cpu_per_socket>1</rasd:cpu_per_socket></Item><Item><rasd:Caption>1024 MB of memory</rasd:Caption><rasd:Description>Memory Size</rasd:Description><rasd:InstanceId>2</rasd:InstanceId><rasd:ResourceType>4</rasd:ResourceType><rasd:AllocationUnits>MegaBytes</rasd:AllocationUnits><rasd:VirtualQuantity>1024</rasd:VirtualQuantity></Item><Item><rasd:Caption>restvm_templates_Disk1</rasd:Caption><rasd:InstanceId>bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d</rasd:InstanceId><rasd:ResourceType>17</rasd:ResourceType><rasd:HostResource>54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d</rasd:HostResource><rasd:Parent>00000000-0000-0000-0000-000000000000</rasd:Parent><rasd:Template>00000000-0000-0000-0000-000000000000</rasd:Template><rasd:ApplicationList></rasd:ApplicationList><rasd:StorageId>8dc966ae-7319-4db8-8fab-33f01a0512b6</rasd:StorageId><rasd:StoragePoolId>ac5d6452-d8fe-4d69-8d98-d99cd5a494bb</rasd:StoragePoolId><rasd:CreationDate>2012/12/04 16:48:47</rasd:CreationDate><rasd:LastModified>2012/12/04 16:48:47</rasd:LastModified><rasd:last_modified_date>2012/12/04 16:48:54</rasd:last_modified_date></Item><Item><rasd:Caption>USB Controller</rasd:Caption><rasd:InstanceId>3</rasd:InstanceId><rasd:ResourceType>23</rasd:ResourceType><rasd:UsbPolicy>DISABLED</rasd:UsbPolicy></Item></Section></Content></ovf:Envelope>\n', 'vm': '6ef07ec2-d19d-47f2-974f-f18bd8760545'}], sdUUID=None, options=None) >Thread-103::DEBUG::2012-12-04 18:48:54,620::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`403408d7-3c2c-4df4-ab5c-5bb889c1058f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-103::DEBUG::2012-12-04 18:48:54,620::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-103::DEBUG::2012-12-04 18:48:54,620::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-103::DEBUG::2012-12-04 18:48:54,621::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`403408d7-3c2c-4df4-ab5c-5bb889c1058f`::Granted request >Thread-103::DEBUG::2012-12-04 18:48:54,621::task::817::TaskManager.Task::(resourceAcquired) Task=`cac8c011-662b-4a04-9bdd-86218b6ff07d`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-103::DEBUG::2012-12-04 18:48:54,621::task::978::TaskManager.Task::(_decref) Task=`cac8c011-662b-4a04-9bdd-86218b6ff07d`::ref 1 aborting False >Thread-103::INFO::2012-12-04 18:48:54,621::sp::1193::Storage.StoragePool::(updateVM) spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-103::INFO::2012-12-04 18:48:54,621::sp::1204::Storage.StoragePool::(updateVM) vmUUID=6ef07ec2-d19d-47f2-974f-f18bd8760545 imgList=['54804768-5a65-49d2-98b6-1a95b7799d8b'] >Thread-103::INFO::2012-12-04 18:48:54,625::logUtils::39::dispatcher::(wrapper) Run and protect: updateVM, Return response: None >Thread-103::DEBUG::2012-12-04 18:48:54,625::task::1172::TaskManager.Task::(prepare) Task=`cac8c011-662b-4a04-9bdd-86218b6ff07d`::finished: None >Thread-103::DEBUG::2012-12-04 18:48:54,626::task::588::TaskManager.Task::(_updateState) Task=`cac8c011-662b-4a04-9bdd-86218b6ff07d`::moving from state preparing -> state finished >Thread-103::DEBUG::2012-12-04 18:48:54,626::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-103::DEBUG::2012-12-04 18:48:54,626::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2012-12-04 18:48:54,626::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-103::DEBUG::2012-12-04 18:48:54,626::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-103::DEBUG::2012-12-04 18:48:54,626::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-103::DEBUG::2012-12-04 18:48:54,626::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-103::DEBUG::2012-12-04 18:48:54,627::task::978::TaskManager.Task::(_decref) Task=`cac8c011-662b-4a04-9bdd-86218b6ff07d`::ref 0 aborting False >Thread-104::DEBUG::2012-12-04 18:48:54,686::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-104::DEBUG::2012-12-04 18:48:54,686::task::588::TaskManager.Task::(_updateState) Task=`b706b53c-e71f-41ca-aa5d-7fe16742dfa5`::moving from state init -> state preparing >Thread-104::INFO::2012-12-04 18:48:54,687::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='d0cfa4da-1b3a-47be-9155-837c4b82b5aa', spUUID=None, options=None) >Thread-104::DEBUG::2012-12-04 18:48:54,687::taskManager::161::TaskManager::(clearTask) Entry. taskID: d0cfa4da-1b3a-47be-9155-837c4b82b5aa >Thread-104::DEBUG::2012-12-04 18:48:54,696::taskManager::166::TaskManager::(clearTask) Return. >Thread-104::INFO::2012-12-04 18:48:54,696::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-104::DEBUG::2012-12-04 18:48:54,696::task::1172::TaskManager.Task::(prepare) Task=`b706b53c-e71f-41ca-aa5d-7fe16742dfa5`::finished: None >Thread-104::DEBUG::2012-12-04 18:48:54,696::task::588::TaskManager.Task::(_updateState) Task=`b706b53c-e71f-41ca-aa5d-7fe16742dfa5`::moving from state preparing -> state finished >Thread-104::DEBUG::2012-12-04 18:48:54,696::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-104::DEBUG::2012-12-04 18:48:54,696::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-104::DEBUG::2012-12-04 18:48:54,697::task::978::TaskManager.Task::(_decref) Task=`b706b53c-e71f-41ca-aa5d-7fe16742dfa5`::ref 0 aborting False >Thread-107::DEBUG::2012-12-04 18:48:57,890::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-107::DEBUG::2012-12-04 18:48:57,890::task::588::TaskManager.Task::(_updateState) Task=`ba494529-ecd9-4af6-9298-21586d801428`::moving from state init -> state preparing >Thread-107::INFO::2012-12-04 18:48:57,890::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-107::INFO::2012-12-04 18:48:57,891::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-107::DEBUG::2012-12-04 18:48:57,891::task::1172::TaskManager.Task::(prepare) Task=`ba494529-ecd9-4af6-9298-21586d801428`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-107::DEBUG::2012-12-04 18:48:57,891::task::588::TaskManager.Task::(_updateState) Task=`ba494529-ecd9-4af6-9298-21586d801428`::moving from state preparing -> state finished >Thread-107::DEBUG::2012-12-04 18:48:57,891::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-107::DEBUG::2012-12-04 18:48:57,891::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-107::DEBUG::2012-12-04 18:48:57,891::task::978::TaskManager.Task::(_decref) Task=`ba494529-ecd9-4af6-9298-21586d801428`::ref 0 aborting False >Thread-108::DEBUG::2012-12-04 18:48:57,977::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-108::DEBUG::2012-12-04 18:48:57,977::task::588::TaskManager.Task::(_updateState) Task=`09a56d72-eba8-4460-987f-33373f2a039b`::moving from state init -> state preparing >Thread-108::INFO::2012-12-04 18:48:57,977::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-108::DEBUG::2012-12-04 18:48:57,977::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`ce6e51c1-d515-4c4b-b49c-db1e2b61a97f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-108::DEBUG::2012-12-04 18:48:57,977::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-108::DEBUG::2012-12-04 18:48:57,978::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-108::DEBUG::2012-12-04 18:48:57,978::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`ce6e51c1-d515-4c4b-b49c-db1e2b61a97f`::Granted request >Thread-108::DEBUG::2012-12-04 18:48:57,978::task::817::TaskManager.Task::(resourceAcquired) Task=`09a56d72-eba8-4460-987f-33373f2a039b`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-108::DEBUG::2012-12-04 18:48:57,978::task::978::TaskManager.Task::(_decref) Task=`09a56d72-eba8-4460-987f-33373f2a039b`::ref 1 aborting False >Thread-108::INFO::2012-12-04 18:48:57,982::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844068028416', 'alerts': [], 'disktotal': '844070125568'}}} >Thread-108::DEBUG::2012-12-04 18:48:57,982::task::1172::TaskManager.Task::(prepare) Task=`09a56d72-eba8-4460-987f-33373f2a039b`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844068028416', 'alerts': [], 'disktotal': '844070125568'}}} >Thread-108::DEBUG::2012-12-04 18:48:57,982::task::588::TaskManager.Task::(_updateState) Task=`09a56d72-eba8-4460-987f-33373f2a039b`::moving from state preparing -> state finished >Thread-108::DEBUG::2012-12-04 18:48:57,982::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-108::DEBUG::2012-12-04 18:48:57,982::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2012-12-04 18:48:57,983::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-108::DEBUG::2012-12-04 18:48:57,983::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-108::DEBUG::2012-12-04 18:48:57,983::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-108::DEBUG::2012-12-04 18:48:57,983::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-108::DEBUG::2012-12-04 18:48:57,983::task::978::TaskManager.Task::(_decref) Task=`09a56d72-eba8-4460-987f-33373f2a039b`::ref 0 aborting False >Thread-109::DEBUG::2012-12-04 18:48:59,025::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-109::DEBUG::2012-12-04 18:48:59,026::task::588::TaskManager.Task::(_updateState) Task=`2fe4e0dc-4cf2-4fde-8171-9fd1b1d89ab4`::moving from state init -> state preparing >Thread-109::INFO::2012-12-04 18:48:59,026::logUtils::37::dispatcher::(wrapper) Run and protect: getVolumeInfo(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', imgUUID='54804768-5a65-49d2-98b6-1a95b7799d8b', volUUID='bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', options=None) >Thread-109::DEBUG::2012-12-04 18:48:59,026::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`c31a02fc-61f0-4230-ba0c-a81d772f5358`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-109::DEBUG::2012-12-04 18:48:59,026::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-109::DEBUG::2012-12-04 18:48:59,026::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-109::DEBUG::2012-12-04 18:48:59,026::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`c31a02fc-61f0-4230-ba0c-a81d772f5358`::Granted request >Thread-109::DEBUG::2012-12-04 18:48:59,027::task::817::TaskManager.Task::(resourceAcquired) Task=`2fe4e0dc-4cf2-4fde-8171-9fd1b1d89ab4`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-109::DEBUG::2012-12-04 18:48:59,027::task::978::TaskManager.Task::(_decref) Task=`2fe4e0dc-4cf2-4fde-8171-9fd1b1d89ab4`::ref 1 aborting False >Thread-109::DEBUG::2012-12-04 18:48:59,028::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-109::INFO::2012-12-04 18:48:59,029::volume::604::Storage.Volume::(getInfo) Info request: sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 imgUUID=54804768-5a65-49d2-98b6-1a95b7799d8b volUUID = bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-109::DEBUG::2012-12-04 18:48:59,043::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-109::DEBUG::2012-12-04 18:48:59,047::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-109::INFO::2012-12-04 18:48:59,050::volume::632::Storage.Volume::(getInfo) 8dc966ae-7319-4db8-8fab-33f01a0512b6/54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d info is {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''} >Thread-109::INFO::2012-12-04 18:48:59,050::logUtils::39::dispatcher::(wrapper) Run and protect: getVolumeInfo, Return response: {'info': {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''}} >Thread-109::DEBUG::2012-12-04 18:48:59,050::task::1172::TaskManager.Task::(prepare) Task=`2fe4e0dc-4cf2-4fde-8171-9fd1b1d89ab4`::finished: {'info': {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''}} >Thread-109::DEBUG::2012-12-04 18:48:59,051::task::588::TaskManager.Task::(_updateState) Task=`2fe4e0dc-4cf2-4fde-8171-9fd1b1d89ab4`::moving from state preparing -> state finished >Thread-109::DEBUG::2012-12-04 18:48:59,051::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >Thread-109::DEBUG::2012-12-04 18:48:59,051::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-109::DEBUG::2012-12-04 18:48:59,051::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-109::DEBUG::2012-12-04 18:48:59,051::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-109::DEBUG::2012-12-04 18:48:59,051::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-109::DEBUG::2012-12-04 18:48:59,051::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-109::DEBUG::2012-12-04 18:48:59,052::task::978::TaskManager.Task::(_decref) Task=`2fe4e0dc-4cf2-4fde-8171-9fd1b1d89ab4`::ref 0 aborting False >Thread-110::DEBUG::2012-12-04 18:48:59,163::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-110::DEBUG::2012-12-04 18:48:59,163::task::588::TaskManager.Task::(_updateState) Task=`77e25ff3-145f-4062-84c2-c80a8b9c3823`::moving from state init -> state preparing >Thread-110::INFO::2012-12-04 18:48:59,163::logUtils::37::dispatcher::(wrapper) Run and protect: getVolumeInfo(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', imgUUID='54804768-5a65-49d2-98b6-1a95b7799d8b', volUUID='bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', options=None) >Thread-110::DEBUG::2012-12-04 18:48:59,163::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`5241a211-7b71-483d-81ff-3e9b38e516ed`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-110::DEBUG::2012-12-04 18:48:59,163::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-110::DEBUG::2012-12-04 18:48:59,164::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-110::DEBUG::2012-12-04 18:48:59,164::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`5241a211-7b71-483d-81ff-3e9b38e516ed`::Granted request >Thread-110::DEBUG::2012-12-04 18:48:59,164::task::817::TaskManager.Task::(resourceAcquired) Task=`77e25ff3-145f-4062-84c2-c80a8b9c3823`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-110::DEBUG::2012-12-04 18:48:59,164::task::978::TaskManager.Task::(_decref) Task=`77e25ff3-145f-4062-84c2-c80a8b9c3823`::ref 1 aborting False >Thread-110::DEBUG::2012-12-04 18:48:59,165::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-110::INFO::2012-12-04 18:48:59,166::volume::604::Storage.Volume::(getInfo) Info request: sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 imgUUID=54804768-5a65-49d2-98b6-1a95b7799d8b volUUID = bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-110::DEBUG::2012-12-04 18:48:59,182::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-110::DEBUG::2012-12-04 18:48:59,190::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-110::INFO::2012-12-04 18:48:59,193::volume::632::Storage.Volume::(getInfo) 8dc966ae-7319-4db8-8fab-33f01a0512b6/54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d info is {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''} >Thread-110::INFO::2012-12-04 18:48:59,193::logUtils::39::dispatcher::(wrapper) Run and protect: getVolumeInfo, Return response: {'info': {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''}} >Thread-110::DEBUG::2012-12-04 18:48:59,193::task::1172::TaskManager.Task::(prepare) Task=`77e25ff3-145f-4062-84c2-c80a8b9c3823`::finished: {'info': {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'LEAF', 'uuid': 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': '', 'children': [], 'ctime': '1354639727', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639727', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '54804768-5a65-49d2-98b6-1a95b7799d8b', 'pool': ''}} >Thread-110::DEBUG::2012-12-04 18:48:59,194::task::588::TaskManager.Task::(_updateState) Task=`77e25ff3-145f-4062-84c2-c80a8b9c3823`::moving from state preparing -> state finished >Thread-110::DEBUG::2012-12-04 18:48:59,194::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >Thread-110::DEBUG::2012-12-04 18:48:59,194::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2012-12-04 18:48:59,194::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-110::DEBUG::2012-12-04 18:48:59,194::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-110::DEBUG::2012-12-04 18:48:59,194::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-110::DEBUG::2012-12-04 18:48:59,195::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-110::DEBUG::2012-12-04 18:48:59,195::task::978::TaskManager.Task::(_decref) Task=`77e25ff3-145f-4062-84c2-c80a8b9c3823`::ref 0 aborting False >Thread-111::DEBUG::2012-12-04 18:48:59,244::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-111::DEBUG::2012-12-04 18:48:59,244::task::588::TaskManager.Task::(_updateState) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::moving from state init -> state preparing >Thread-111::INFO::2012-12-04 18:48:59,244::logUtils::37::dispatcher::(wrapper) Run and protect: copyImage(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', vmUUID='', srcImgUUID='54804768-5a65-49d2-98b6-1a95b7799d8b', srcVolUUID='bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', dstImgUUID='7a7da037-71d4-45f3-99d6-c99721c3ddb3', dstVolUUID='07a18903-93c1-44fb-a610-60f744cae7b2', description='Active VM', dstSdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', volType=6, volFormat=4, preallocate=2, postZero='false', force='false') >Thread-111::INFO::2012-12-04 18:48:59,244::fileSD::260::Storage.StorageDomain::(validate) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-111::DEBUG::2012-12-04 18:48:59,247::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=49966f75e3c1291add4e8ecc44a5ac665a2d7fe0'] >Thread-111::DEBUG::2012-12-04 18:48:59,249::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-111::INFO::2012-12-04 18:48:59,256::image::470::Storage.Image::(isLegal) image 54804768-5a65-49d2-98b6-1a95b7799d8b in domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 has vollist ['bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d'] >Thread-111::DEBUG::2012-12-04 18:48:59,256::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-111::DEBUG::2012-12-04 18:48:59,269::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-111::DEBUG::2012-12-04 18:48:59,273::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-111::DEBUG::2012-12-04 18:48:59,277::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-111::INFO::2012-12-04 18:48:59,283::image::357::Storage.Image::(getChain) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 imgUUID=54804768-5a65-49d2-98b6-1a95b7799d8b chain=[<storage.fileVolume.FileVolume instance at 0x7f7f140ecdd0>] >Thread-111::DEBUG::2012-12-04 18:48:59,286::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`bbe715f2-de05-49b4-955b-d01737575130`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-111::DEBUG::2012-12-04 18:48:59,286::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-111::DEBUG::2012-12-04 18:48:59,286::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-111::DEBUG::2012-12-04 18:48:59,286::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`bbe715f2-de05-49b4-955b-d01737575130`::Granted request >Thread-111::DEBUG::2012-12-04 18:48:59,286::task::817::TaskManager.Task::(resourceAcquired) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-111::DEBUG::2012-12-04 18:48:59,286::task::978::TaskManager.Task::(_decref) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::ref 1 aborting False >Thread-111::DEBUG::2012-12-04 18:48:59,290::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >Thread-111::DEBUG::2012-12-04 18:48:59,309::taskManager::66::TaskManager::(scheduleJob) scheduled job copyImage for task 2889e958-982c-4dc6-9d54-1893fe68a8b1 >Thread-111::INFO::2012-12-04 18:48:59,310::logUtils::39::dispatcher::(wrapper) Run and protect: copyImage, Return response: None >Thread-111::DEBUG::2012-12-04 18:48:59,310::task::1167::TaskManager.Task::(prepare) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::Prepare: 1 jobs exist, move to acquiring >Thread-111::DEBUG::2012-12-04 18:48:59,310::task::588::TaskManager.Task::(_updateState) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::moving from state preparing -> state acquiring >Thread-111::DEBUG::2012-12-04 18:48:59,311::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >Thread-111::DEBUG::2012-12-04 18:48:59,324::task::588::TaskManager.Task::(_updateState) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::moving from state acquiring -> state queued >Thread-111::DEBUG::2012-12-04 18:48:59,325::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >Thread-111::DEBUG::2012-12-04 18:48:59,348::taskManager::48::TaskManager::(_queueTask) queuing task: 2889e958-982c-4dc6-9d54-1893fe68a8b1 >Thread-111::DEBUG::2012-12-04 18:48:59,349::taskManager::54::TaskManager::(_queueTask) task queued: 2889e958-982c-4dc6-9d54-1893fe68a8b1 >Thread-111::DEBUG::2012-12-04 18:48:59,349::task::1169::TaskManager.Task::(prepare) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::returning >d0cfa4da-1b3a-47be-9155-837c4b82b5aa::DEBUG::2012-12-04 18:48:59,349::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-111::DEBUG::2012-12-04 18:48:59,349::task::978::TaskManager.Task::(_decref) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::ref 0 aborting False >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,350::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: 2889e958-982c-4dc6-9d54-1893fe68a8b1 running: <bound method Task.commit of <storage.task.Task instance at 0x7f7f2c278758>> with: None >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,350::task::1181::TaskManager.Task::(commit) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::committing task: 2889e958-982c-4dc6-9d54-1893fe68a8b1 >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,350::task::588::TaskManager.Task::(_updateState) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::moving from state queued -> state running >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,351::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,365::task::892::TaskManager.Task::(_runJobs) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::Task.run: running job 0: copyImage: <bound method StoragePool.copyImage of <storage.sp.StoragePool object at 0x7f7f2c193e10>> (args: ('8dc966ae-7319-4db8-8fab-33f01a0512b6', '', '54804768-5a65-49d2-98b6-1a95b7799d8b', 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', '7a7da037-71d4-45f3-99d6-c99721c3ddb3', '07a18903-93c1-44fb-a610-60f744cae7b2', 'Active VM', '8dc966ae-7319-4db8-8fab-33f01a0512b6', 6, 4, 2, False, False) kwargs: {}) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,366::task::317::TaskManager.Task::(run) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::Job.run: running copyImage: <bound method StoragePool.copyImage of <storage.sp.StoragePool object at 0x7f7f2c193e10>> (args: ('8dc966ae-7319-4db8-8fab-33f01a0512b6', '', '54804768-5a65-49d2-98b6-1a95b7799d8b', 'bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', '7a7da037-71d4-45f3-99d6-c99721c3ddb3', '07a18903-93c1-44fb-a610-60f744cae7b2', 'Active VM', '8dc966ae-7319-4db8-8fab-33f01a0512b6', 6, 4, 2, False, False) kwargs: {}) callback None >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,366::resourceManager::175::ResourceManager.Request::(__init__) ResName=`8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b`ReqID=`128eeff9-8e55-4b22-94d2-18fb15acebb7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,366::resourceManager::486::ResourceManager::(registerResource) Trying to register resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' for lock type 'shared' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,368::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,374::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,381::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,389::image::357::Storage.Image::(getChain) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 imgUUID=54804768-5a65-49d2-98b6-1a95b7799d8b chain=[<storage.fileVolume.FileVolume instance at 0x7f7f2c2787a0>] >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,391::resourceManager::175::ResourceManager.Request::(__init__) ResName=`8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS.bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d`ReqID=`068dcc39-e5a0-4d92-9683-a5a31cdf235c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,391::resourceManager::486::ResourceManager::(registerResource) Trying to register resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS.bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d' for lock type 'shared' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,392::resourceManager::528::ResourceManager::(registerResource) Resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS.bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d' is free. Now locking as 'shared' (1 active user) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,392::resourceManager::212::ResourceManager.Request::(grant) ResName=`8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS.bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d`ReqID=`068dcc39-e5a0-4d92-9683-a5a31cdf235c`::Granted request >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,392::resourceManager::528::ResourceManager::(registerResource) Resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' is free. Now locking as 'shared' (1 active user) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,392::resourceManager::212::ResourceManager.Request::(grant) ResName=`8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b`ReqID=`128eeff9-8e55-4b22-94d2-18fb15acebb7`::Granted request >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,393::resourceManager::175::ResourceManager.Request::(__init__) ResName=`8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.7a7da037-71d4-45f3-99d6-c99721c3ddb3`ReqID=`c8ae999f-0325-4280-9332-ba4fd0d2f37a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,393::resourceManager::486::ResourceManager::(registerResource) Trying to register resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.7a7da037-71d4-45f3-99d6-c99721c3ddb3' for lock type 'exclusive' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,394::resourceFactories::119::Storage.ResourcesFactories::(__getResourceCandidatesList) Image 7a7da037-71d4-45f3-99d6-c99721c3ddb3 does not exist in domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,394::resourceManager::528::ResourceManager::(registerResource) Resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.7a7da037-71d4-45f3-99d6-c99721c3ddb3' is free. Now locking as 'exclusive' (1 active user) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,394::resourceManager::212::ResourceManager.Request::(grant) ResName=`8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.7a7da037-71d4-45f3-99d6-c99721c3ddb3`ReqID=`c8ae999f-0325-4280-9332-ba4fd0d2f37a`::Granted request >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,394::image::725::Storage.Image::(copy) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 vmUUID= srcImgUUID=54804768-5a65-49d2-98b6-1a95b7799d8b srcVolUUID=bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d dstImgUUID=7a7da037-71d4-45f3-99d6-c99721c3ddb3 dstVolUUID=07a18903-93c1-44fb-a610-60f744cae7b2 dstSdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 volType=6 volFormat=COW preallocate=SPARSE force=False postZero=False >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,395::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,423::image::770::Storage.Image::(copy) copy source 8dc966ae-7319-4db8-8fab-33f01a0512b6:54804768-5a65-49d2-98b6-1a95b7799d8b:bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d vol size 4194304 destination 8dc966ae-7319-4db8-8fab-33f01a0512b6:7a7da037-71d4-45f3-99d6-c99721c3ddb3:07a18903-93c1-44fb-a610-60f744cae7b2 apparentsize 385 >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,423::image::470::Storage.Image::(isLegal) image 7a7da037-71d4-45f3-99d6-c99721c3ddb3 in domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 has vollist [] >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,424::image::94::Storage.Image::(create) Create placeholder /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/7a7da037-71d4-45f3-99d6-c99721c3ddb3 for image's volumes >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,425::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,448::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,474::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,502::fileVolume::171::Storage.Volume::(create) fileVolume: create: volUUID 07a18903-93c1-44fb-a610-60f744cae7b2 srcImg 00000000-0000-0000-0000-000000000000 srvVol 00000000-0000-0000-0000-000000000000 >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,502::fileVolume::174::Storage.Volume::(create) Request to create COW volume /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2 with size = 20480 sectors >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,502::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/qemu-img create -f qcow2 /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2 10240K' (cwd None) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,529::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,531::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,592::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,623::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,632::volume::521::Storage.Volume::(prepare) Volume: preparing volume 8dc966ae-7319-4db8-8fab-33f01a0512b6/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:48:59,637::volume::521::Storage.Volume::(prepare) Volume: preparing volume 8dc966ae-7319-4db8-8fab-33f01a0512b6/07a18903-93c1-44fb-a610-60f744cae7b2 >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,665::volume::813::Storage.Volume::(qemuConvert) (qemuConvert): COPY /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d (qcow2) to /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2 (qcow2) START >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,665::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/nice -n 19 /usr/bin/ionice -c 2 -n 7 /usr/bin/qemu-img convert -t none -f qcow2 /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d -O qcow2 /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2' (cwd None) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:48:59,670::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,227::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = []; <rc> = 0 >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,228::volume::825::Storage.Volume::(qemuConvert) (qemuConvert): COPY /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/54804768-5a65-49d2-98b6-1a95b7799d8b/bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d to /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6/images/7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2 DONE >2889e958-982c-4dc6-9d54-1893fe68a8b1::INFO::2012-12-04 18:49:00,241::image::838::Storage.Image::(copy) Finished copying 8dc966ae-7319-4db8-8fab-33f01a0512b6:bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d -> 8dc966ae-7319-4db8-8fab-33f01a0512b6:07a18903-93c1-44fb-a610-60f744cae7b2 >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,241::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.7a7da037-71d4-45f3-99d6-c99721c3ddb3' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,241::resourceManager::553::ResourceManager::(releaseResource) Released resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.7a7da037-71d4-45f3-99d6-c99721c3ddb3' (0 active users) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,241::resourceManager::558::ResourceManager::(releaseResource) Resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.7a7da037-71d4-45f3-99d6-c99721c3ddb3' is free, finding out if anyone is waiting for it. >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,241::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.7a7da037-71d4-45f3-99d6-c99721c3ddb3', Clearing records. >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,242::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,242::resourceManager::553::ResourceManager::(releaseResource) Released resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' (0 active users) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,242::resourceManager::558::ResourceManager::(releaseResource) Resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b' is free, finding out if anyone is waiting for it. >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,242::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS.bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,242::resourceManager::553::ResourceManager::(releaseResource) Released resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS.bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d' (0 active users) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,243::resourceManager::558::ResourceManager::(releaseResource) Resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS.bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d' is free, finding out if anyone is waiting for it. >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,243::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS.bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d', Clearing records. >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,243::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource '8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS.54804768-5a65-49d2-98b6-1a95b7799d8b', Clearing records. >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,243::task::588::TaskManager.Task::(_updateState) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::moving from state running -> state finished >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,244::task::740::TaskManager.Task::(_save) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/2889e958-982c-4dc6-9d54-1893fe68a8b1.temp >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,287::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,287::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,287::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,288::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,288::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,288::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,288::task::900::TaskManager.Task::(_runJobs) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::Task.run: exit - success: result {'uuid': '07a18903-93c1-44fb-a610-60f744cae7b2'} >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,288::task::978::TaskManager.Task::(_decref) Task=`2889e958-982c-4dc6-9d54-1893fe68a8b1`::ref 0 aborting False >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:49:00,288::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-113::DEBUG::2012-12-04 18:49:01,909::task::588::TaskManager.Task::(_updateState) Task=`850f1243-f371-4f0c-bdca-28666eb2202f`::moving from state init -> state preparing >Thread-113::INFO::2012-12-04 18:49:01,909::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2012-12-04 18:49:01,909::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00213503837585', 'lastCheck': 1354639736.202152, 'code': 0, 'valid': True}} >Thread-113::DEBUG::2012-12-04 18:49:01,909::task::1172::TaskManager.Task::(prepare) Task=`850f1243-f371-4f0c-bdca-28666eb2202f`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00213503837585', 'lastCheck': 1354639736.202152, 'code': 0, 'valid': True}} >Thread-113::DEBUG::2012-12-04 18:49:01,909::task::588::TaskManager.Task::(_updateState) Task=`850f1243-f371-4f0c-bdca-28666eb2202f`::moving from state preparing -> state finished >Thread-113::DEBUG::2012-12-04 18:49:01,910::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2012-12-04 18:49:01,910::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2012-12-04 18:49:01,910::task::978::TaskManager.Task::(_decref) Task=`850f1243-f371-4f0c-bdca-28666eb2202f`::ref 0 aborting False >Thread-116::DEBUG::2012-12-04 18:49:04,326::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-116::DEBUG::2012-12-04 18:49:04,326::task::588::TaskManager.Task::(_updateState) Task=`4c62f4a2-f65c-4b46-a9ec-58365c176757`::moving from state init -> state preparing >Thread-116::INFO::2012-12-04 18:49:04,326::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-116::DEBUG::2012-12-04 18:49:04,326::taskManager::103::TaskManager::(getAllTasksStatuses) Entry. >Thread-116::DEBUG::2012-12-04 18:49:04,326::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 2889e958-982c-4dc6-9d54-1893fe68a8b1 >Thread-116::DEBUG::2012-12-04 18:49:04,326::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '2889e958-982c-4dc6-9d54-1893fe68a8b1'} >Thread-116::DEBUG::2012-12-04 18:49:04,327::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {'2889e958-982c-4dc6-9d54-1893fe68a8b1': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '2889e958-982c-4dc6-9d54-1893fe68a8b1'}} >Thread-116::INFO::2012-12-04 18:49:04,327::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'2889e958-982c-4dc6-9d54-1893fe68a8b1': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '2889e958-982c-4dc6-9d54-1893fe68a8b1'}}} >Thread-116::DEBUG::2012-12-04 18:49:04,327::task::1172::TaskManager.Task::(prepare) Task=`4c62f4a2-f65c-4b46-a9ec-58365c176757`::finished: {'allTasksStatus': {'2889e958-982c-4dc6-9d54-1893fe68a8b1': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '2889e958-982c-4dc6-9d54-1893fe68a8b1'}}} >Thread-116::DEBUG::2012-12-04 18:49:04,327::task::588::TaskManager.Task::(_updateState) Task=`4c62f4a2-f65c-4b46-a9ec-58365c176757`::moving from state preparing -> state finished >Thread-116::DEBUG::2012-12-04 18:49:04,327::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-116::DEBUG::2012-12-04 18:49:04,327::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-116::DEBUG::2012-12-04 18:49:04,327::task::978::TaskManager.Task::(_decref) Task=`4c62f4a2-f65c-4b46-a9ec-58365c176757`::ref 0 aborting False >Thread-117::DEBUG::2012-12-04 18:49:04,434::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-117::DEBUG::2012-12-04 18:49:04,434::task::588::TaskManager.Task::(_updateState) Task=`6957823b-e975-406f-982e-219592aef682`::moving from state init -> state preparing >Thread-117::INFO::2012-12-04 18:49:04,434::logUtils::37::dispatcher::(wrapper) Run and protect: getVolumeInfo(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', imgUUID='7a7da037-71d4-45f3-99d6-c99721c3ddb3', volUUID='07a18903-93c1-44fb-a610-60f744cae7b2', options=None) >Thread-117::DEBUG::2012-12-04 18:49:04,435::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`b72edafd-ae3e-4d70-b1be-0b6b1cc7f688`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-117::DEBUG::2012-12-04 18:49:04,435::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-117::DEBUG::2012-12-04 18:49:04,435::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-117::DEBUG::2012-12-04 18:49:04,435::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`b72edafd-ae3e-4d70-b1be-0b6b1cc7f688`::Granted request >Thread-117::DEBUG::2012-12-04 18:49:04,435::task::817::TaskManager.Task::(resourceAcquired) Task=`6957823b-e975-406f-982e-219592aef682`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-117::DEBUG::2012-12-04 18:49:04,436::task::978::TaskManager.Task::(_decref) Task=`6957823b-e975-406f-982e-219592aef682`::ref 1 aborting False >Thread-117::DEBUG::2012-12-04 18:49:04,438::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-117::INFO::2012-12-04 18:49:04,440::volume::604::Storage.Volume::(getInfo) Info request: sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 imgUUID=7a7da037-71d4-45f3-99d6-c99721c3ddb3 volUUID = 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-117::DEBUG::2012-12-04 18:49:04,456::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-117::DEBUG::2012-12-04 18:49:04,460::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-117::INFO::2012-12-04 18:49:04,463::volume::632::Storage.Volume::(getInfo) 8dc966ae-7319-4db8-8fab-33f01a0512b6/7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2 info is {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'SHARED', 'uuid': '07a18903-93c1-44fb-a610-60f744cae7b2', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': 'Active VM', 'children': [], 'ctime': '1354639739', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639739', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '7a7da037-71d4-45f3-99d6-c99721c3ddb3', 'pool': ''} >Thread-117::INFO::2012-12-04 18:49:04,463::logUtils::39::dispatcher::(wrapper) Run and protect: getVolumeInfo, Return response: {'info': {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'SHARED', 'uuid': '07a18903-93c1-44fb-a610-60f744cae7b2', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': 'Active VM', 'children': [], 'ctime': '1354639739', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639739', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '7a7da037-71d4-45f3-99d6-c99721c3ddb3', 'pool': ''}} >Thread-117::DEBUG::2012-12-04 18:49:04,463::task::1172::TaskManager.Task::(prepare) Task=`6957823b-e975-406f-982e-219592aef682`::finished: {'info': {'status': 'OK', 'domain': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'truesize': '512', 'voltype': 'SHARED', 'uuid': '07a18903-93c1-44fb-a610-60f744cae7b2', 'parent': '00000000-0000-0000-0000-000000000000', 'format': 'COW', 'description': 'Active VM', 'children': [], 'ctime': '1354639739', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '1354639739', 'capacity': '2147483648', 'apparentsize': '197120', 'type': 'SPARSE', 'image': '7a7da037-71d4-45f3-99d6-c99721c3ddb3', 'pool': ''}} >Thread-117::DEBUG::2012-12-04 18:49:04,464::task::588::TaskManager.Task::(_updateState) Task=`6957823b-e975-406f-982e-219592aef682`::moving from state preparing -> state finished >Thread-117::DEBUG::2012-12-04 18:49:04,464::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >Thread-117::DEBUG::2012-12-04 18:49:04,464::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-117::DEBUG::2012-12-04 18:49:04,464::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-117::DEBUG::2012-12-04 18:49:04,464::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-117::DEBUG::2012-12-04 18:49:04,464::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-117::DEBUG::2012-12-04 18:49:04,465::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-117::DEBUG::2012-12-04 18:49:04,465::task::978::TaskManager.Task::(_decref) Task=`6957823b-e975-406f-982e-219592aef682`::ref 0 aborting False >Thread-118::DEBUG::2012-12-04 18:49:04,610::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-118::DEBUG::2012-12-04 18:49:04,610::task::588::TaskManager.Task::(_updateState) Task=`3451ec59-88cb-4bfd-92c4-2d63ba79f859`::moving from state init -> state preparing >Thread-118::INFO::2012-12-04 18:49:04,611::logUtils::37::dispatcher::(wrapper) Run and protect: updateVM(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', vmList=[{'imglist': '7a7da037-71d4-45f3-99d6-c99721c3ddb3', 'ovf': '<?xml version="1.0" encoding="UTF-8"?><ovf:Envelope xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1/" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ovf:version="3.2.0.0"><References><File ovf:href="7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2" ovf:id="07a18903-93c1-44fb-a610-60f744cae7b2" ovf:size="2147483648" ovf:description=""></File></References><Section xsi:type="ovf:NetworkSection_Type"><Info>List of networks</Info><Network ovf:name="Network 1"></Network></Section><Section xsi:type="ovf:DiskSection_Type"><Info>List of Virtual Disks</Info><Disk ovf:diskId="07a18903-93c1-44fb-a610-60f744cae7b2" ovf:size="2" ovf:actual_size="0" ovf:vm_snapshot_id="fbc405d8-651f-4b13-b58a-23ec3e3f4825" ovf:parentRef="" ovf:fileRef="7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2" ovf:format="http://www.gnome.org/~markmc/qcow-image-format.html" ovf:volume-format="COW" ovf:volume-type="Sparse" ovf:disk-interface="IDE" ovf:boot="false" ovf:disk-alias="restvm_templates_Disk1" ovf:wipe-after-delete="false"></Disk></Section><Content ovf:id="out" xsi:type="ovf:VirtualSystem_Type"><Description></Description><Domain></Domain><CreationDate>2012/12/04 16:48:59</CreationDate><ExportDate>2012/12/04 16:49:04</ExportDate><IsAutoSuspend>false</IsAutoSuspend><IsSmartcardEnabled>false</IsSmartcardEnabled><TimeZone></TimeZone><default_boot_sequence>0</default_boot_sequence><VmType>0</VmType><Name>rest_templ3</Name><TemplateId>282107d0-01d7-4cbc-891a-77eda089aae8</TemplateId><Origin>3</Origin><default_display_type>1</default_display_type><Section ovf:id="282107d0-01d7-4cbc-891a-77eda089aae8" ovf:required="false" xsi:type="ovf:OperatingSystemSection_Type"><Info>Guest Operating System</Info><Description>Unassigned</Description></Section><Section xsi:type="ovf:VirtualHardwareSection_Type"><Info>1 CPU, 1024 Memeory</Info><System><vssd:VirtualSystemType>ENGINE 3.2.0.0</vssd:VirtualSystemType></System><Item><rasd:Caption>1 virtual cpu</rasd:Caption><rasd:Description>Number of virtual CPU</rasd:Description><rasd:InstanceId>1</rasd:InstanceId><rasd:ResourceType>3</rasd:ResourceType><rasd:num_of_sockets>1</rasd:num_of_sockets><rasd:cpu_per_socket>1</rasd:cpu_per_socket></Item><Item><rasd:Caption>1024 MB of memory</rasd:Caption><rasd:Description>Memory Size</rasd:Description><rasd:InstanceId>2</rasd:InstanceId><rasd:ResourceType>4</rasd:ResourceType><rasd:AllocationUnits>MegaBytes</rasd:AllocationUnits><rasd:VirtualQuantity>1024</rasd:VirtualQuantity></Item><Item><rasd:Caption>restvm_templates_Disk1</rasd:Caption><rasd:InstanceId>07a18903-93c1-44fb-a610-60f744cae7b2</rasd:InstanceId><rasd:ResourceType>17</rasd:ResourceType><rasd:HostResource>7a7da037-71d4-45f3-99d6-c99721c3ddb3/07a18903-93c1-44fb-a610-60f744cae7b2</rasd:HostResource><rasd:Parent>00000000-0000-0000-0000-000000000000</rasd:Parent><rasd:Template>00000000-0000-0000-0000-000000000000</rasd:Template><rasd:ApplicationList></rasd:ApplicationList><rasd:StorageId>8dc966ae-7319-4db8-8fab-33f01a0512b6</rasd:StorageId><rasd:StoragePoolId>ac5d6452-d8fe-4d69-8d98-d99cd5a494bb</rasd:StoragePoolId><rasd:CreationDate>2012/12/04 16:48:59</rasd:CreationDate><rasd:LastModified>2012/12/04 16:48:59</rasd:LastModified></Item><Item><rasd:Caption>USB Controller</rasd:Caption><rasd:InstanceId>3</rasd:InstanceId><rasd:ResourceType>23</rasd:ResourceType><rasd:UsbPolicy>DISABLED</rasd:UsbPolicy></Item></Section></Content></ovf:Envelope>\n', 'vm': '282107d0-01d7-4cbc-891a-77eda089aae8'}], sdUUID=None, options=None) >Thread-118::DEBUG::2012-12-04 18:49:04,611::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`04a3191b-be2c-4c92-90c3-307ec501e85a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-118::DEBUG::2012-12-04 18:49:04,611::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-118::DEBUG::2012-12-04 18:49:04,611::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-118::DEBUG::2012-12-04 18:49:04,611::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`04a3191b-be2c-4c92-90c3-307ec501e85a`::Granted request >Thread-118::DEBUG::2012-12-04 18:49:04,612::task::817::TaskManager.Task::(resourceAcquired) Task=`3451ec59-88cb-4bfd-92c4-2d63ba79f859`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-118::DEBUG::2012-12-04 18:49:04,612::task::978::TaskManager.Task::(_decref) Task=`3451ec59-88cb-4bfd-92c4-2d63ba79f859`::ref 1 aborting False >Thread-118::INFO::2012-12-04 18:49:04,612::sp::1193::Storage.StoragePool::(updateVM) spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-118::INFO::2012-12-04 18:49:04,612::sp::1204::Storage.StoragePool::(updateVM) vmUUID=282107d0-01d7-4cbc-891a-77eda089aae8 imgList=['7a7da037-71d4-45f3-99d6-c99721c3ddb3'] >Thread-118::INFO::2012-12-04 18:49:04,615::logUtils::39::dispatcher::(wrapper) Run and protect: updateVM, Return response: None >Thread-118::DEBUG::2012-12-04 18:49:04,615::task::1172::TaskManager.Task::(prepare) Task=`3451ec59-88cb-4bfd-92c4-2d63ba79f859`::finished: None >Thread-118::DEBUG::2012-12-04 18:49:04,615::task::588::TaskManager.Task::(_updateState) Task=`3451ec59-88cb-4bfd-92c4-2d63ba79f859`::moving from state preparing -> state finished >Thread-118::DEBUG::2012-12-04 18:49:04,616::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-118::DEBUG::2012-12-04 18:49:04,616::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-118::DEBUG::2012-12-04 18:49:04,616::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-118::DEBUG::2012-12-04 18:49:04,616::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-118::DEBUG::2012-12-04 18:49:04,616::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-118::DEBUG::2012-12-04 18:49:04,616::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-118::DEBUG::2012-12-04 18:49:04,617::task::978::TaskManager.Task::(_decref) Task=`3451ec59-88cb-4bfd-92c4-2d63ba79f859`::ref 0 aborting False >Thread-119::DEBUG::2012-12-04 18:49:04,700::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-119::DEBUG::2012-12-04 18:49:04,700::task::588::TaskManager.Task::(_updateState) Task=`a62c3bd1-5020-4ca4-8cb4-efce8dceb42f`::moving from state init -> state preparing >Thread-119::INFO::2012-12-04 18:49:04,700::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='2889e958-982c-4dc6-9d54-1893fe68a8b1', spUUID=None, options=None) >Thread-119::DEBUG::2012-12-04 18:49:04,701::taskManager::161::TaskManager::(clearTask) Entry. taskID: 2889e958-982c-4dc6-9d54-1893fe68a8b1 >Thread-119::DEBUG::2012-12-04 18:49:04,714::taskManager::166::TaskManager::(clearTask) Return. >Thread-119::INFO::2012-12-04 18:49:04,715::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-119::DEBUG::2012-12-04 18:49:04,715::task::1172::TaskManager.Task::(prepare) Task=`a62c3bd1-5020-4ca4-8cb4-efce8dceb42f`::finished: None >Thread-119::DEBUG::2012-12-04 18:49:04,715::task::588::TaskManager.Task::(_updateState) Task=`a62c3bd1-5020-4ca4-8cb4-efce8dceb42f`::moving from state preparing -> state finished >Thread-119::DEBUG::2012-12-04 18:49:04,715::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-119::DEBUG::2012-12-04 18:49:04,715::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-119::DEBUG::2012-12-04 18:49:04,715::task::978::TaskManager.Task::(_decref) Task=`a62c3bd1-5020-4ca4-8cb4-efce8dceb42f`::ref 0 aborting False >Thread-121::DEBUG::2012-12-04 18:49:08,021::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-121::DEBUG::2012-12-04 18:49:08,022::task::588::TaskManager.Task::(_updateState) Task=`e0f4e8d9-d681-450b-bb5a-e0f6b9d2f671`::moving from state init -> state preparing >Thread-121::INFO::2012-12-04 18:49:08,022::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-121::INFO::2012-12-04 18:49:08,022::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-121::DEBUG::2012-12-04 18:49:08,022::task::1172::TaskManager.Task::(prepare) Task=`e0f4e8d9-d681-450b-bb5a-e0f6b9d2f671`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-121::DEBUG::2012-12-04 18:49:08,022::task::588::TaskManager.Task::(_updateState) Task=`e0f4e8d9-d681-450b-bb5a-e0f6b9d2f671`::moving from state preparing -> state finished >Thread-121::DEBUG::2012-12-04 18:49:08,022::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-121::DEBUG::2012-12-04 18:49:08,023::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-121::DEBUG::2012-12-04 18:49:08,023::task::978::TaskManager.Task::(_decref) Task=`e0f4e8d9-d681-450b-bb5a-e0f6b9d2f671`::ref 0 aborting False >Thread-122::DEBUG::2012-12-04 18:49:08,054::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-122::DEBUG::2012-12-04 18:49:08,054::task::588::TaskManager.Task::(_updateState) Task=`79c47873-8451-4be1-b6ed-6f6eef091b69`::moving from state init -> state preparing >Thread-122::INFO::2012-12-04 18:49:08,055::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-122::DEBUG::2012-12-04 18:49:08,055::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`fba48179-10f3-4ebb-aba8-d7e83dab685e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-122::DEBUG::2012-12-04 18:49:08,055::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-122::DEBUG::2012-12-04 18:49:08,055::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-122::DEBUG::2012-12-04 18:49:08,055::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`fba48179-10f3-4ebb-aba8-d7e83dab685e`::Granted request >Thread-122::DEBUG::2012-12-04 18:49:08,056::task::817::TaskManager.Task::(resourceAcquired) Task=`79c47873-8451-4be1-b6ed-6f6eef091b69`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-122::DEBUG::2012-12-04 18:49:08,056::task::978::TaskManager.Task::(_decref) Task=`79c47873-8451-4be1-b6ed-6f6eef091b69`::ref 1 aborting False >Thread-122::INFO::2012-12-04 18:49:08,059::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844068028416', 'alerts': [], 'disktotal': '844070125568'}}} >Thread-122::DEBUG::2012-12-04 18:49:08,059::task::1172::TaskManager.Task::(prepare) Task=`79c47873-8451-4be1-b6ed-6f6eef091b69`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844068028416', 'alerts': [], 'disktotal': '844070125568'}}} >Thread-122::DEBUG::2012-12-04 18:49:08,060::task::588::TaskManager.Task::(_updateState) Task=`79c47873-8451-4be1-b6ed-6f6eef091b69`::moving from state preparing -> state finished >Thread-122::DEBUG::2012-12-04 18:49:08,060::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-122::DEBUG::2012-12-04 18:49:08,060::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2012-12-04 18:49:08,060::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-122::DEBUG::2012-12-04 18:49:08,060::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-122::DEBUG::2012-12-04 18:49:08,060::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-122::DEBUG::2012-12-04 18:49:08,060::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-122::DEBUG::2012-12-04 18:49:08,061::task::978::TaskManager.Task::(_decref) Task=`79c47873-8451-4be1-b6ed-6f6eef091b69`::ref 0 aborting False >Thread-125::DEBUG::2012-12-04 18:49:12,310::task::588::TaskManager.Task::(_updateState) Task=`191b78c8-dbe1-4a9b-a6db-f66f2120cd66`::moving from state init -> state preparing >Thread-125::INFO::2012-12-04 18:49:12,310::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-125::INFO::2012-12-04 18:49:12,310::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00294899940491', 'lastCheck': 1354639746.210917, 'code': 0, 'valid': True}} >Thread-125::DEBUG::2012-12-04 18:49:12,311::task::1172::TaskManager.Task::(prepare) Task=`191b78c8-dbe1-4a9b-a6db-f66f2120cd66`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00294899940491', 'lastCheck': 1354639746.210917, 'code': 0, 'valid': True}} >Thread-125::DEBUG::2012-12-04 18:49:12,311::task::588::TaskManager.Task::(_updateState) Task=`191b78c8-dbe1-4a9b-a6db-f66f2120cd66`::moving from state preparing -> state finished >Thread-125::DEBUG::2012-12-04 18:49:12,311::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-125::DEBUG::2012-12-04 18:49:12,311::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-125::DEBUG::2012-12-04 18:49:12,311::task::978::TaskManager.Task::(_decref) Task=`191b78c8-dbe1-4a9b-a6db-f66f2120cd66`::ref 0 aborting False >Thread-129::DEBUG::2012-12-04 18:49:18,097::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-129::DEBUG::2012-12-04 18:49:18,098::task::588::TaskManager.Task::(_updateState) Task=`cbc870d3-73bb-48eb-8618-d591a446db4a`::moving from state init -> state preparing >Thread-129::INFO::2012-12-04 18:49:18,098::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-129::INFO::2012-12-04 18:49:18,098::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-129::DEBUG::2012-12-04 18:49:18,098::task::1172::TaskManager.Task::(prepare) Task=`cbc870d3-73bb-48eb-8618-d591a446db4a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-129::DEBUG::2012-12-04 18:49:18,098::task::588::TaskManager.Task::(_updateState) Task=`cbc870d3-73bb-48eb-8618-d591a446db4a`::moving from state preparing -> state finished >Thread-129::DEBUG::2012-12-04 18:49:18,099::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-129::DEBUG::2012-12-04 18:49:18,099::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-129::DEBUG::2012-12-04 18:49:18,099::task::978::TaskManager.Task::(_decref) Task=`cbc870d3-73bb-48eb-8618-d591a446db4a`::ref 0 aborting False >Thread-130::DEBUG::2012-12-04 18:49:18,145::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-130::DEBUG::2012-12-04 18:49:18,145::task::588::TaskManager.Task::(_updateState) Task=`4d7ddfd5-cfb9-4af0-836f-c94381a2f4e8`::moving from state init -> state preparing >Thread-130::INFO::2012-12-04 18:49:18,146::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-130::DEBUG::2012-12-04 18:49:18,146::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`359b1747-b998-4233-837b-afa3f9e18271`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-130::DEBUG::2012-12-04 18:49:18,146::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-130::DEBUG::2012-12-04 18:49:18,146::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-130::DEBUG::2012-12-04 18:49:18,146::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`359b1747-b998-4233-837b-afa3f9e18271`::Granted request >Thread-130::DEBUG::2012-12-04 18:49:18,147::task::817::TaskManager.Task::(resourceAcquired) Task=`4d7ddfd5-cfb9-4af0-836f-c94381a2f4e8`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-130::DEBUG::2012-12-04 18:49:18,147::task::978::TaskManager.Task::(_decref) Task=`4d7ddfd5-cfb9-4af0-836f-c94381a2f4e8`::ref 1 aborting False >Thread-130::INFO::2012-12-04 18:49:18,151::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844009308160', 'alerts': [], 'disktotal': '844014551040'}}} >Thread-130::DEBUG::2012-12-04 18:49:18,151::task::1172::TaskManager.Task::(prepare) Task=`4d7ddfd5-cfb9-4af0-836f-c94381a2f4e8`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844009308160', 'alerts': [], 'disktotal': '844014551040'}}} >Thread-130::DEBUG::2012-12-04 18:49:18,151::task::588::TaskManager.Task::(_updateState) Task=`4d7ddfd5-cfb9-4af0-836f-c94381a2f4e8`::moving from state preparing -> state finished >Thread-130::DEBUG::2012-12-04 18:49:18,151::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-130::DEBUG::2012-12-04 18:49:18,151::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2012-12-04 18:49:18,151::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-130::DEBUG::2012-12-04 18:49:18,151::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-130::DEBUG::2012-12-04 18:49:18,152::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-130::DEBUG::2012-12-04 18:49:18,152::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-130::DEBUG::2012-12-04 18:49:18,152::task::978::TaskManager.Task::(_decref) Task=`4d7ddfd5-cfb9-4af0-836f-c94381a2f4e8`::ref 0 aborting False >Thread-133::DEBUG::2012-12-04 18:49:22,679::task::588::TaskManager.Task::(_updateState) Task=`906515e5-66d9-4308-92f7-8a93b3d83754`::moving from state init -> state preparing >Thread-133::INFO::2012-12-04 18:49:22,679::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2012-12-04 18:49:22,679::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0142929553986', 'lastCheck': 1354639757.468281, 'code': 0, 'valid': True}} >Thread-133::DEBUG::2012-12-04 18:49:22,679::task::1172::TaskManager.Task::(prepare) Task=`906515e5-66d9-4308-92f7-8a93b3d83754`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0142929553986', 'lastCheck': 1354639757.468281, 'code': 0, 'valid': True}} >Thread-133::DEBUG::2012-12-04 18:49:22,679::task::588::TaskManager.Task::(_updateState) Task=`906515e5-66d9-4308-92f7-8a93b3d83754`::moving from state preparing -> state finished >Thread-133::DEBUG::2012-12-04 18:49:22,680::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2012-12-04 18:49:22,680::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2012-12-04 18:49:22,680::task::978::TaskManager.Task::(_decref) Task=`906515e5-66d9-4308-92f7-8a93b3d83754`::ref 0 aborting False >Thread-137::DEBUG::2012-12-04 18:49:28,218::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-137::DEBUG::2012-12-04 18:49:28,218::task::588::TaskManager.Task::(_updateState) Task=`fe928c16-7730-4711-bda5-9596f4aac292`::moving from state init -> state preparing >Thread-137::INFO::2012-12-04 18:49:28,219::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-137::INFO::2012-12-04 18:49:28,219::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-137::DEBUG::2012-12-04 18:49:28,219::task::1172::TaskManager.Task::(prepare) Task=`fe928c16-7730-4711-bda5-9596f4aac292`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-137::DEBUG::2012-12-04 18:49:28,219::task::588::TaskManager.Task::(_updateState) Task=`fe928c16-7730-4711-bda5-9596f4aac292`::moving from state preparing -> state finished >Thread-137::DEBUG::2012-12-04 18:49:28,219::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-137::DEBUG::2012-12-04 18:49:28,219::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-137::DEBUG::2012-12-04 18:49:28,219::task::978::TaskManager.Task::(_decref) Task=`fe928c16-7730-4711-bda5-9596f4aac292`::ref 0 aborting False >Thread-138::DEBUG::2012-12-04 18:49:28,286::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-138::DEBUG::2012-12-04 18:49:28,287::task::588::TaskManager.Task::(_updateState) Task=`76c08a7c-f845-4fcf-ae18-378efd0c0890`::moving from state init -> state preparing >Thread-138::INFO::2012-12-04 18:49:28,287::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-138::DEBUG::2012-12-04 18:49:28,287::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`53540a0a-22b3-4b60-97c0-d44d9da5ed2a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-138::DEBUG::2012-12-04 18:49:28,287::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-138::DEBUG::2012-12-04 18:49:28,287::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-138::DEBUG::2012-12-04 18:49:28,288::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`53540a0a-22b3-4b60-97c0-d44d9da5ed2a`::Granted request >Thread-138::DEBUG::2012-12-04 18:49:28,288::task::817::TaskManager.Task::(resourceAcquired) Task=`76c08a7c-f845-4fcf-ae18-378efd0c0890`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-138::DEBUG::2012-12-04 18:49:28,288::task::978::TaskManager.Task::(_decref) Task=`76c08a7c-f845-4fcf-ae18-378efd0c0890`::ref 1 aborting False >Thread-138::INFO::2012-12-04 18:49:28,292::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844360581120', 'alerts': [], 'disktotal': '844365824000'}}} >Thread-138::DEBUG::2012-12-04 18:49:28,292::task::1172::TaskManager.Task::(prepare) Task=`76c08a7c-f845-4fcf-ae18-378efd0c0890`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844360581120', 'alerts': [], 'disktotal': '844365824000'}}} >Thread-138::DEBUG::2012-12-04 18:49:28,292::task::588::TaskManager.Task::(_updateState) Task=`76c08a7c-f845-4fcf-ae18-378efd0c0890`::moving from state preparing -> state finished >Thread-138::DEBUG::2012-12-04 18:49:28,292::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-138::DEBUG::2012-12-04 18:49:28,292::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-138::DEBUG::2012-12-04 18:49:28,292::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-138::DEBUG::2012-12-04 18:49:28,293::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-138::DEBUG::2012-12-04 18:49:28,293::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-138::DEBUG::2012-12-04 18:49:28,293::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-138::DEBUG::2012-12-04 18:49:28,293::task::978::TaskManager.Task::(_decref) Task=`76c08a7c-f845-4fcf-ae18-378efd0c0890`::ref 0 aborting False >Thread-141::DEBUG::2012-12-04 18:49:33,218::task::588::TaskManager.Task::(_updateState) Task=`7691e9c4-9f4a-4237-9142-d5db69815a5a`::moving from state init -> state preparing >Thread-141::INFO::2012-12-04 18:49:33,218::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-141::INFO::2012-12-04 18:49:33,219::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0023410320282', 'lastCheck': 1354639767.476294, 'code': 0, 'valid': True}} >Thread-141::DEBUG::2012-12-04 18:49:33,219::task::1172::TaskManager.Task::(prepare) Task=`7691e9c4-9f4a-4237-9142-d5db69815a5a`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0023410320282', 'lastCheck': 1354639767.476294, 'code': 0, 'valid': True}} >Thread-141::DEBUG::2012-12-04 18:49:33,219::task::588::TaskManager.Task::(_updateState) Task=`7691e9c4-9f4a-4237-9142-d5db69815a5a`::moving from state preparing -> state finished >Thread-141::DEBUG::2012-12-04 18:49:33,219::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-141::DEBUG::2012-12-04 18:49:33,220::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-141::DEBUG::2012-12-04 18:49:33,220::task::978::TaskManager.Task::(_decref) Task=`7691e9c4-9f4a-4237-9142-d5db69815a5a`::ref 0 aborting False >Thread-145::DEBUG::2012-12-04 18:49:38,419::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-145::DEBUG::2012-12-04 18:49:38,420::task::588::TaskManager.Task::(_updateState) Task=`e8ea5534-b7e6-43f8-84d9-3fc1ce66a15b`::moving from state init -> state preparing >Thread-145::INFO::2012-12-04 18:49:38,420::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-145::INFO::2012-12-04 18:49:38,420::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-145::DEBUG::2012-12-04 18:49:38,420::task::1172::TaskManager.Task::(prepare) Task=`e8ea5534-b7e6-43f8-84d9-3fc1ce66a15b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-145::DEBUG::2012-12-04 18:49:38,420::task::588::TaskManager.Task::(_updateState) Task=`e8ea5534-b7e6-43f8-84d9-3fc1ce66a15b`::moving from state preparing -> state finished >Thread-145::DEBUG::2012-12-04 18:49:38,420::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-145::DEBUG::2012-12-04 18:49:38,420::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-145::DEBUG::2012-12-04 18:49:38,421::task::978::TaskManager.Task::(_decref) Task=`e8ea5534-b7e6-43f8-84d9-3fc1ce66a15b`::ref 0 aborting False >Thread-146::DEBUG::2012-12-04 18:49:38,483::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-146::DEBUG::2012-12-04 18:49:38,483::task::588::TaskManager.Task::(_updateState) Task=`73db4cf3-ca7d-4516-9d02-822d15af5b05`::moving from state init -> state preparing >Thread-146::INFO::2012-12-04 18:49:38,483::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-146::DEBUG::2012-12-04 18:49:38,484::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`5ac711a5-f53d-4a12-8593-b62202e5aa07`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-146::DEBUG::2012-12-04 18:49:38,484::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-146::DEBUG::2012-12-04 18:49:38,484::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-146::DEBUG::2012-12-04 18:49:38,484::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`5ac711a5-f53d-4a12-8593-b62202e5aa07`::Granted request >Thread-146::DEBUG::2012-12-04 18:49:38,484::task::817::TaskManager.Task::(resourceAcquired) Task=`73db4cf3-ca7d-4516-9d02-822d15af5b05`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-146::DEBUG::2012-12-04 18:49:38,484::task::978::TaskManager.Task::(_decref) Task=`73db4cf3-ca7d-4516-9d02-822d15af5b05`::ref 1 aborting False >Thread-146::INFO::2012-12-04 18:49:38,488::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844360581120', 'alerts': [], 'disktotal': '844365824000'}}} >Thread-146::DEBUG::2012-12-04 18:49:38,488::task::1172::TaskManager.Task::(prepare) Task=`73db4cf3-ca7d-4516-9d02-822d15af5b05`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844360581120', 'alerts': [], 'disktotal': '844365824000'}}} >Thread-146::DEBUG::2012-12-04 18:49:38,488::task::588::TaskManager.Task::(_updateState) Task=`73db4cf3-ca7d-4516-9d02-822d15af5b05`::moving from state preparing -> state finished >Thread-146::DEBUG::2012-12-04 18:49:38,489::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-146::DEBUG::2012-12-04 18:49:38,489::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-146::DEBUG::2012-12-04 18:49:38,489::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-146::DEBUG::2012-12-04 18:49:38,489::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-146::DEBUG::2012-12-04 18:49:38,489::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-146::DEBUG::2012-12-04 18:49:38,489::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-146::DEBUG::2012-12-04 18:49:38,490::task::978::TaskManager.Task::(_decref) Task=`73db4cf3-ca7d-4516-9d02-822d15af5b05`::ref 0 aborting False >Thread-149::DEBUG::2012-12-04 18:49:43,767::task::588::TaskManager.Task::(_updateState) Task=`a5f82f56-8330-4d7a-be94-578aeac72758`::moving from state init -> state preparing >Thread-149::INFO::2012-12-04 18:49:43,767::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-149::INFO::2012-12-04 18:49:43,768::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00226616859436', 'lastCheck': 1354639777.484015, 'code': 0, 'valid': True}} >Thread-149::DEBUG::2012-12-04 18:49:43,768::task::1172::TaskManager.Task::(prepare) Task=`a5f82f56-8330-4d7a-be94-578aeac72758`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00226616859436', 'lastCheck': 1354639777.484015, 'code': 0, 'valid': True}} >Thread-149::DEBUG::2012-12-04 18:49:43,768::task::588::TaskManager.Task::(_updateState) Task=`a5f82f56-8330-4d7a-be94-578aeac72758`::moving from state preparing -> state finished >Thread-149::DEBUG::2012-12-04 18:49:43,768::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-149::DEBUG::2012-12-04 18:49:43,768::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-149::DEBUG::2012-12-04 18:49:43,768::task::978::TaskManager.Task::(_decref) Task=`a5f82f56-8330-4d7a-be94-578aeac72758`::ref 0 aborting False >Thread-153::DEBUG::2012-12-04 18:49:48,647::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-153::DEBUG::2012-12-04 18:49:48,647::task::588::TaskManager.Task::(_updateState) Task=`6507bd7e-0c5c-4fe8-a88f-307b01fe1836`::moving from state init -> state preparing >Thread-153::INFO::2012-12-04 18:49:48,648::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-153::INFO::2012-12-04 18:49:48,648::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-153::DEBUG::2012-12-04 18:49:48,648::task::1172::TaskManager.Task::(prepare) Task=`6507bd7e-0c5c-4fe8-a88f-307b01fe1836`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-153::DEBUG::2012-12-04 18:49:48,648::task::588::TaskManager.Task::(_updateState) Task=`6507bd7e-0c5c-4fe8-a88f-307b01fe1836`::moving from state preparing -> state finished >Thread-153::DEBUG::2012-12-04 18:49:48,648::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-153::DEBUG::2012-12-04 18:49:48,648::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-153::DEBUG::2012-12-04 18:49:48,648::task::978::TaskManager.Task::(_decref) Task=`6507bd7e-0c5c-4fe8-a88f-307b01fe1836`::ref 0 aborting False >Thread-154::DEBUG::2012-12-04 18:49:48,685::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-154::DEBUG::2012-12-04 18:49:48,685::task::588::TaskManager.Task::(_updateState) Task=`076d298e-f06f-4551-8a66-0fb067de805f`::moving from state init -> state preparing >Thread-154::INFO::2012-12-04 18:49:48,685::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-154::DEBUG::2012-12-04 18:49:48,686::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`4fc075a0-cad7-4d02-8c62-6ec5f3924bb7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-154::DEBUG::2012-12-04 18:49:48,686::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-154::DEBUG::2012-12-04 18:49:48,686::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-154::DEBUG::2012-12-04 18:49:48,686::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`4fc075a0-cad7-4d02-8c62-6ec5f3924bb7`::Granted request >Thread-154::DEBUG::2012-12-04 18:49:48,686::task::817::TaskManager.Task::(resourceAcquired) Task=`076d298e-f06f-4551-8a66-0fb067de805f`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-154::DEBUG::2012-12-04 18:49:48,687::task::978::TaskManager.Task::(_decref) Task=`076d298e-f06f-4551-8a66-0fb067de805f`::ref 1 aborting False >Thread-154::INFO::2012-12-04 18:49:48,690::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844398329856', 'alerts': [], 'disktotal': '844403572736'}}} >Thread-154::DEBUG::2012-12-04 18:49:48,690::task::1172::TaskManager.Task::(prepare) Task=`076d298e-f06f-4551-8a66-0fb067de805f`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844398329856', 'alerts': [], 'disktotal': '844403572736'}}} >Thread-154::DEBUG::2012-12-04 18:49:48,691::task::588::TaskManager.Task::(_updateState) Task=`076d298e-f06f-4551-8a66-0fb067de805f`::moving from state preparing -> state finished >Thread-154::DEBUG::2012-12-04 18:49:48,691::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-154::DEBUG::2012-12-04 18:49:48,691::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-154::DEBUG::2012-12-04 18:49:48,691::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-154::DEBUG::2012-12-04 18:49:48,691::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-154::DEBUG::2012-12-04 18:49:48,691::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-154::DEBUG::2012-12-04 18:49:48,691::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-154::DEBUG::2012-12-04 18:49:48,692::task::978::TaskManager.Task::(_decref) Task=`076d298e-f06f-4551-8a66-0fb067de805f`::ref 0 aborting False >Thread-157::DEBUG::2012-12-04 18:49:54,304::task::588::TaskManager.Task::(_updateState) Task=`721493fa-8497-4a3e-a504-629c7e8c9e31`::moving from state init -> state preparing >Thread-157::INFO::2012-12-04 18:49:54,304::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-157::INFO::2012-12-04 18:49:54,305::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00265598297119', 'lastCheck': 1354639787.492903, 'code': 0, 'valid': True}} >Thread-157::DEBUG::2012-12-04 18:49:54,305::task::1172::TaskManager.Task::(prepare) Task=`721493fa-8497-4a3e-a504-629c7e8c9e31`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00265598297119', 'lastCheck': 1354639787.492903, 'code': 0, 'valid': True}} >Thread-157::DEBUG::2012-12-04 18:49:54,305::task::588::TaskManager.Task::(_updateState) Task=`721493fa-8497-4a3e-a504-629c7e8c9e31`::moving from state preparing -> state finished >Thread-157::DEBUG::2012-12-04 18:49:54,305::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-157::DEBUG::2012-12-04 18:49:54,305::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-157::DEBUG::2012-12-04 18:49:54,305::task::978::TaskManager.Task::(_decref) Task=`721493fa-8497-4a3e-a504-629c7e8c9e31`::ref 0 aborting False >Thread-161::DEBUG::2012-12-04 18:49:58,732::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-161::DEBUG::2012-12-04 18:49:58,732::task::588::TaskManager.Task::(_updateState) Task=`2984c1fb-dc2a-4823-9031-83da676bee88`::moving from state init -> state preparing >Thread-161::INFO::2012-12-04 18:49:58,732::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-161::INFO::2012-12-04 18:49:58,732::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-161::DEBUG::2012-12-04 18:49:58,732::task::1172::TaskManager.Task::(prepare) Task=`2984c1fb-dc2a-4823-9031-83da676bee88`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-161::DEBUG::2012-12-04 18:49:58,733::task::588::TaskManager.Task::(_updateState) Task=`2984c1fb-dc2a-4823-9031-83da676bee88`::moving from state preparing -> state finished >Thread-161::DEBUG::2012-12-04 18:49:58,733::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-161::DEBUG::2012-12-04 18:49:58,733::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-161::DEBUG::2012-12-04 18:49:58,733::task::978::TaskManager.Task::(_decref) Task=`2984c1fb-dc2a-4823-9031-83da676bee88`::ref 0 aborting False >Thread-162::DEBUG::2012-12-04 18:49:58,774::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-162::DEBUG::2012-12-04 18:49:58,774::task::588::TaskManager.Task::(_updateState) Task=`e4ece9b3-f695-47b8-95f9-ae985bdb1e52`::moving from state init -> state preparing >Thread-162::INFO::2012-12-04 18:49:58,774::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-162::DEBUG::2012-12-04 18:49:58,775::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`a1c22288-3211-494d-91b2-b371d175e9c4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-162::DEBUG::2012-12-04 18:49:58,775::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-162::DEBUG::2012-12-04 18:49:58,775::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-162::DEBUG::2012-12-04 18:49:58,775::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`a1c22288-3211-494d-91b2-b371d175e9c4`::Granted request >Thread-162::DEBUG::2012-12-04 18:49:58,776::task::817::TaskManager.Task::(resourceAcquired) Task=`e4ece9b3-f695-47b8-95f9-ae985bdb1e52`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-162::DEBUG::2012-12-04 18:49:58,776::task::978::TaskManager.Task::(_decref) Task=`e4ece9b3-f695-47b8-95f9-ae985bdb1e52`::ref 1 aborting False >Thread-162::INFO::2012-12-04 18:49:58,780::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844661522432', 'alerts': [], 'disktotal': '844665716736'}}} >Thread-162::DEBUG::2012-12-04 18:49:58,780::task::1172::TaskManager.Task::(prepare) Task=`e4ece9b3-f695-47b8-95f9-ae985bdb1e52`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844661522432', 'alerts': [], 'disktotal': '844665716736'}}} >Thread-162::DEBUG::2012-12-04 18:49:58,780::task::588::TaskManager.Task::(_updateState) Task=`e4ece9b3-f695-47b8-95f9-ae985bdb1e52`::moving from state preparing -> state finished >Thread-162::DEBUG::2012-12-04 18:49:58,780::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-162::DEBUG::2012-12-04 18:49:58,780::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-162::DEBUG::2012-12-04 18:49:58,780::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-162::DEBUG::2012-12-04 18:49:58,781::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-162::DEBUG::2012-12-04 18:49:58,781::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-162::DEBUG::2012-12-04 18:49:58,781::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-162::DEBUG::2012-12-04 18:49:58,781::task::978::TaskManager.Task::(_decref) Task=`e4ece9b3-f695-47b8-95f9-ae985bdb1e52`::ref 0 aborting False >Thread-165::DEBUG::2012-12-04 18:50:03,607::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-165::DEBUG::2012-12-04 18:50:03,607::task::588::TaskManager.Task::(_updateState) Task=`3bf52df3-3b7e-4a74-9e46-885770c85ca6`::moving from state init -> state preparing >Thread-165::INFO::2012-12-04 18:50:03,607::logUtils::37::dispatcher::(wrapper) Run and protect: removeVM(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', vmUUID='6ef07ec2-d19d-47f2-974f-f18bd8760545', sdUUID=None, options=None) >Thread-165::DEBUG::2012-12-04 18:50:03,607::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`8ff809fe-afc6-4001-91eb-b1388ad229f4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-165::DEBUG::2012-12-04 18:50:03,608::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-165::DEBUG::2012-12-04 18:50:03,608::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-165::DEBUG::2012-12-04 18:50:03,608::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`8ff809fe-afc6-4001-91eb-b1388ad229f4`::Granted request >Thread-165::DEBUG::2012-12-04 18:50:03,608::task::817::TaskManager.Task::(resourceAcquired) Task=`3bf52df3-3b7e-4a74-9e46-885770c85ca6`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-165::DEBUG::2012-12-04 18:50:03,608::task::978::TaskManager.Task::(_decref) Task=`3bf52df3-3b7e-4a74-9e46-885770c85ca6`::ref 1 aborting False >Thread-165::INFO::2012-12-04 18:50:03,609::sp::1228::Storage.StoragePool::(removeVM) spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb vmUUID=6ef07ec2-d19d-47f2-974f-f18bd8760545 sdUUID=None >Thread-165::INFO::2012-12-04 18:50:03,618::logUtils::39::dispatcher::(wrapper) Run and protect: removeVM, Return response: None >Thread-165::DEBUG::2012-12-04 18:50:03,618::task::1172::TaskManager.Task::(prepare) Task=`3bf52df3-3b7e-4a74-9e46-885770c85ca6`::finished: None >Thread-165::DEBUG::2012-12-04 18:50:03,619::task::588::TaskManager.Task::(_updateState) Task=`3bf52df3-3b7e-4a74-9e46-885770c85ca6`::moving from state preparing -> state finished >Thread-165::DEBUG::2012-12-04 18:50:03,619::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-165::DEBUG::2012-12-04 18:50:03,619::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-165::DEBUG::2012-12-04 18:50:03,619::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-165::DEBUG::2012-12-04 18:50:03,619::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-165::DEBUG::2012-12-04 18:50:03,619::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-165::DEBUG::2012-12-04 18:50:03,619::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-165::DEBUG::2012-12-04 18:50:03,620::task::978::TaskManager.Task::(_decref) Task=`3bf52df3-3b7e-4a74-9e46-885770c85ca6`::ref 0 aborting False >Thread-166::DEBUG::2012-12-04 18:50:03,857::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-166::DEBUG::2012-12-04 18:50:03,857::task::588::TaskManager.Task::(_updateState) Task=`4fc01025-ba5b-40ba-9cad-82c6fd02a5d4`::moving from state init -> state preparing >Thread-166::INFO::2012-12-04 18:50:03,857::logUtils::37::dispatcher::(wrapper) Run and protect: removeVM(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', vmUUID='2707a35d-fa83-46d9-aec8-dc23fc1ba0a8', sdUUID=None, options=None) >Thread-166::DEBUG::2012-12-04 18:50:03,858::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`05c12399-832c-4f17-8309-311ad2af141c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-166::DEBUG::2012-12-04 18:50:03,858::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-166::DEBUG::2012-12-04 18:50:03,858::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-166::DEBUG::2012-12-04 18:50:03,858::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`05c12399-832c-4f17-8309-311ad2af141c`::Granted request >Thread-166::DEBUG::2012-12-04 18:50:03,859::task::817::TaskManager.Task::(resourceAcquired) Task=`4fc01025-ba5b-40ba-9cad-82c6fd02a5d4`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-166::DEBUG::2012-12-04 18:50:03,859::task::978::TaskManager.Task::(_decref) Task=`4fc01025-ba5b-40ba-9cad-82c6fd02a5d4`::ref 1 aborting False >Thread-166::INFO::2012-12-04 18:50:03,859::sp::1228::Storage.StoragePool::(removeVM) spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb vmUUID=2707a35d-fa83-46d9-aec8-dc23fc1ba0a8 sdUUID=None >Thread-166::INFO::2012-12-04 18:50:03,860::logUtils::39::dispatcher::(wrapper) Run and protect: removeVM, Return response: None >Thread-166::DEBUG::2012-12-04 18:50:03,860::task::1172::TaskManager.Task::(prepare) Task=`4fc01025-ba5b-40ba-9cad-82c6fd02a5d4`::finished: None >Thread-166::DEBUG::2012-12-04 18:50:03,860::task::588::TaskManager.Task::(_updateState) Task=`4fc01025-ba5b-40ba-9cad-82c6fd02a5d4`::moving from state preparing -> state finished >Thread-166::DEBUG::2012-12-04 18:50:03,860::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-166::DEBUG::2012-12-04 18:50:03,860::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-166::DEBUG::2012-12-04 18:50:03,860::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-166::DEBUG::2012-12-04 18:50:03,861::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-166::DEBUG::2012-12-04 18:50:03,861::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-166::DEBUG::2012-12-04 18:50:03,861::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-166::DEBUG::2012-12-04 18:50:03,861::task::978::TaskManager.Task::(_decref) Task=`4fc01025-ba5b-40ba-9cad-82c6fd02a5d4`::ref 0 aborting False >Thread-167::DEBUG::2012-12-04 18:50:04,022::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-167::DEBUG::2012-12-04 18:50:04,022::task::588::TaskManager.Task::(_updateState) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::moving from state init -> state preparing >Thread-167::INFO::2012-12-04 18:50:04,022::logUtils::37::dispatcher::(wrapper) Run and protect: deleteImage(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', imgUUID='54804768-5a65-49d2-98b6-1a95b7799d8b', postZero='false', force='false') >Thread-167::INFO::2012-12-04 18:50:04,022::fileSD::260::Storage.StorageDomain::(validate) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-167::DEBUG::2012-12-04 18:50:04,025::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=49966f75e3c1291add4e8ecc44a5ac665a2d7fe0'] >Thread-167::DEBUG::2012-12-04 18:50:04,025::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.54804768-5a65-49d2-98b6-1a95b7799d8b`ReqID=`961142fb-bc38-46ae-b5a8-213d287801dd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-167::DEBUG::2012-12-04 18:50:04,026::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.54804768-5a65-49d2-98b6-1a95b7799d8b' for lock type 'exclusive' >Thread-167::DEBUG::2012-12-04 18:50:04,026::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.54804768-5a65-49d2-98b6-1a95b7799d8b' is free. Now locking as 'exclusive' (1 active user) >Thread-167::DEBUG::2012-12-04 18:50:04,026::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.54804768-5a65-49d2-98b6-1a95b7799d8b`ReqID=`961142fb-bc38-46ae-b5a8-213d287801dd`::Granted request >Thread-167::DEBUG::2012-12-04 18:50:04,026::task::817::TaskManager.Task::(resourceAcquired) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::_resourcesAcquired: Storage.54804768-5a65-49d2-98b6-1a95b7799d8b (exclusive) >Thread-167::DEBUG::2012-12-04 18:50:04,026::task::978::TaskManager.Task::(_decref) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::ref 1 aborting False >Thread-167::DEBUG::2012-12-04 18:50:04,027::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`8a9f7033-b250-45fd-a974-48b4ad62f225`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-167::DEBUG::2012-12-04 18:50:04,027::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-167::DEBUG::2012-12-04 18:50:04,027::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-167::DEBUG::2012-12-04 18:50:04,027::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`8a9f7033-b250-45fd-a974-48b4ad62f225`::Granted request >Thread-167::DEBUG::2012-12-04 18:50:04,027::task::817::TaskManager.Task::(resourceAcquired) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-167::DEBUG::2012-12-04 18:50:04,028::task::978::TaskManager.Task::(_decref) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::ref 1 aborting False >Thread-167::DEBUG::2012-12-04 18:50:04,030::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-167::DEBUG::2012-12-04 18:50:04,033::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-167::DEBUG::2012-12-04 18:50:04,040::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-167::DEBUG::2012-12-04 18:50:04,044::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-167::DEBUG::2012-12-04 18:50:04,047::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d >Thread-167::INFO::2012-12-04 18:50:04,050::volume::420::Storage.Volume::(setLegality) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 imgUUID=54804768-5a65-49d2-98b6-1a95b7799d8b volUUID = bc29ad0b-bfb6-4ea6-8e57-f3562907dd7d legality = ILLEGAL >Thread-167::DEBUG::2012-12-04 18:50:04,063::task::740::TaskManager.Task::(_save) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c.temp >Thread-167::DEBUG::2012-12-04 18:50:04,073::taskManager::66::TaskManager::(scheduleJob) scheduled job deleteImage for task b534ef17-f251-464a-a83b-b67dcc95922c >Thread-167::INFO::2012-12-04 18:50:04,073::logUtils::39::dispatcher::(wrapper) Run and protect: deleteImage, Return response: None >Thread-167::DEBUG::2012-12-04 18:50:04,073::task::1167::TaskManager.Task::(prepare) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::Prepare: 1 jobs exist, move to acquiring >Thread-167::DEBUG::2012-12-04 18:50:04,073::task::588::TaskManager.Task::(_updateState) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::moving from state preparing -> state acquiring >Thread-167::DEBUG::2012-12-04 18:50:04,074::task::740::TaskManager.Task::(_save) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c.temp >Thread-167::DEBUG::2012-12-04 18:50:04,086::task::588::TaskManager.Task::(_updateState) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::moving from state acquiring -> state queued >Thread-167::DEBUG::2012-12-04 18:50:04,087::task::740::TaskManager.Task::(_save) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c.temp >Thread-167::DEBUG::2012-12-04 18:50:04,100::taskManager::48::TaskManager::(_queueTask) queuing task: b534ef17-f251-464a-a83b-b67dcc95922c >Thread-167::DEBUG::2012-12-04 18:50:04,100::taskManager::54::TaskManager::(_queueTask) task queued: b534ef17-f251-464a-a83b-b67dcc95922c >Thread-7::DEBUG::2012-12-04 18:50:04,100::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-167::DEBUG::2012-12-04 18:50:04,101::task::1169::TaskManager.Task::(prepare) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::returning >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,101::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: b534ef17-f251-464a-a83b-b67dcc95922c running: <bound method Task.commit of <storage.task.Task instance at 0x7f7f140ec518>> with: None >Thread-167::DEBUG::2012-12-04 18:50:04,101::task::978::TaskManager.Task::(_decref) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::ref 0 aborting False >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,101::task::1181::TaskManager.Task::(commit) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::committing task: b534ef17-f251-464a-a83b-b67dcc95922c >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,102::task::588::TaskManager.Task::(_updateState) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::moving from state queued -> state running >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,102::task::740::TaskManager.Task::(_save) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c.temp >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,115::task::892::TaskManager.Task::(_runJobs) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::Task.run: running job 0: deleteImage: <function <lambda> at 0x7f7f140e5f50> (args: () kwargs: {}) >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,115::task::317::TaskManager.Task::(run) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::Job.run: running deleteImage: <function <lambda> at 0x7f7f140e5f50> (args: () kwargs: {}) callback None >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,116::task::588::TaskManager.Task::(_updateState) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::moving from state running -> state finished >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,116::task::740::TaskManager.Task::(_save) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/b534ef17-f251-464a-a83b-b67dcc95922c.temp >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,131::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.54804768-5a65-49d2-98b6-1a95b7799d8b': < ResourceRef 'Storage.54804768-5a65-49d2-98b6-1a95b7799d8b', isValid: 'True' obj: 'None'>, 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,131::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,131::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.54804768-5a65-49d2-98b6-1a95b7799d8b' >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,131::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.54804768-5a65-49d2-98b6-1a95b7799d8b' (0 active users) >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,131::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.54804768-5a65-49d2-98b6-1a95b7799d8b' is free, finding out if anyone is waiting for it. >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,131::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.54804768-5a65-49d2-98b6-1a95b7799d8b', Clearing records. >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,132::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,132::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,132::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,132::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,132::task::900::TaskManager.Task::(_runJobs) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::Task.run: exit - success: result True >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,132::task::978::TaskManager.Task::(_decref) Task=`b534ef17-f251-464a-a83b-b67dcc95922c`::ref 0 aborting False >b534ef17-f251-464a-a83b-b67dcc95922c::DEBUG::2012-12-04 18:50:04,133::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-168::DEBUG::2012-12-04 18:50:05,063::task::588::TaskManager.Task::(_updateState) Task=`ea3956f3-aada-4ab7-8ce9-946231f9a237`::moving from state init -> state preparing >Thread-168::INFO::2012-12-04 18:50:05,063::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-168::INFO::2012-12-04 18:50:05,063::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00231313705444', 'lastCheck': 1354639797.500969, 'code': 0, 'valid': True}} >Thread-168::DEBUG::2012-12-04 18:50:05,063::task::1172::TaskManager.Task::(prepare) Task=`ea3956f3-aada-4ab7-8ce9-946231f9a237`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00231313705444', 'lastCheck': 1354639797.500969, 'code': 0, 'valid': True}} >Thread-168::DEBUG::2012-12-04 18:50:05,063::task::588::TaskManager.Task::(_updateState) Task=`ea3956f3-aada-4ab7-8ce9-946231f9a237`::moving from state preparing -> state finished >Thread-168::DEBUG::2012-12-04 18:50:05,063::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-168::DEBUG::2012-12-04 18:50:05,063::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-168::DEBUG::2012-12-04 18:50:05,064::task::978::TaskManager.Task::(_decref) Task=`ea3956f3-aada-4ab7-8ce9-946231f9a237`::ref 0 aborting False >Thread-171::DEBUG::2012-12-04 18:50:08,828::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-171::DEBUG::2012-12-04 18:50:08,828::task::588::TaskManager.Task::(_updateState) Task=`5214fa8f-9083-4655-8a44-9616e4a27e80`::moving from state init -> state preparing >Thread-171::INFO::2012-12-04 18:50:08,829::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-171::INFO::2012-12-04 18:50:08,829::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-171::DEBUG::2012-12-04 18:50:08,829::task::1172::TaskManager.Task::(prepare) Task=`5214fa8f-9083-4655-8a44-9616e4a27e80`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-171::DEBUG::2012-12-04 18:50:08,829::task::588::TaskManager.Task::(_updateState) Task=`5214fa8f-9083-4655-8a44-9616e4a27e80`::moving from state preparing -> state finished >Thread-171::DEBUG::2012-12-04 18:50:08,829::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-171::DEBUG::2012-12-04 18:50:08,829::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-171::DEBUG::2012-12-04 18:50:08,830::task::978::TaskManager.Task::(_decref) Task=`5214fa8f-9083-4655-8a44-9616e4a27e80`::ref 0 aborting False >Thread-172::DEBUG::2012-12-04 18:50:08,856::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-172::DEBUG::2012-12-04 18:50:08,856::task::588::TaskManager.Task::(_updateState) Task=`6fbe6ce8-a699-40d4-a878-a595d33c4305`::moving from state init -> state preparing >Thread-172::INFO::2012-12-04 18:50:08,856::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-172::DEBUG::2012-12-04 18:50:08,856::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`c8cf4b29-9183-4fcf-9cd0-3b4d531c61ce`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-172::DEBUG::2012-12-04 18:50:08,857::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-172::DEBUG::2012-12-04 18:50:08,857::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-172::DEBUG::2012-12-04 18:50:08,857::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`c8cf4b29-9183-4fcf-9cd0-3b4d531c61ce`::Granted request >Thread-172::DEBUG::2012-12-04 18:50:08,857::task::817::TaskManager.Task::(resourceAcquired) Task=`6fbe6ce8-a699-40d4-a878-a595d33c4305`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-172::DEBUG::2012-12-04 18:50:08,857::task::978::TaskManager.Task::(_decref) Task=`6fbe6ce8-a699-40d4-a878-a595d33c4305`::ref 1 aborting False >Thread-172::INFO::2012-12-04 18:50:08,861::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844661522432', 'alerts': [], 'disktotal': '844665716736'}}} >Thread-172::DEBUG::2012-12-04 18:50:08,861::task::1172::TaskManager.Task::(prepare) Task=`6fbe6ce8-a699-40d4-a878-a595d33c4305`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844661522432', 'alerts': [], 'disktotal': '844665716736'}}} >Thread-172::DEBUG::2012-12-04 18:50:08,861::task::588::TaskManager.Task::(_updateState) Task=`6fbe6ce8-a699-40d4-a878-a595d33c4305`::moving from state preparing -> state finished >Thread-172::DEBUG::2012-12-04 18:50:08,861::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-172::DEBUG::2012-12-04 18:50:08,861::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-172::DEBUG::2012-12-04 18:50:08,862::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-172::DEBUG::2012-12-04 18:50:08,862::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-172::DEBUG::2012-12-04 18:50:08,862::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-172::DEBUG::2012-12-04 18:50:08,862::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-172::DEBUG::2012-12-04 18:50:08,862::task::978::TaskManager.Task::(_decref) Task=`6fbe6ce8-a699-40d4-a878-a595d33c4305`::ref 0 aborting False >Thread-176::DEBUG::2012-12-04 18:50:14,394::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-176::DEBUG::2012-12-04 18:50:14,394::task::588::TaskManager.Task::(_updateState) Task=`6dfa2d51-04bf-4d79-bda2-307d36f1bf74`::moving from state init -> state preparing >Thread-176::INFO::2012-12-04 18:50:14,394::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-176::DEBUG::2012-12-04 18:50:14,394::taskManager::103::TaskManager::(getAllTasksStatuses) Entry. >Thread-176::DEBUG::2012-12-04 18:50:14,394::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: b534ef17-f251-464a-a83b-b67dcc95922c >Thread-176::DEBUG::2012-12-04 18:50:14,394::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'b534ef17-f251-464a-a83b-b67dcc95922c'} >Thread-176::DEBUG::2012-12-04 18:50:14,395::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {'b534ef17-f251-464a-a83b-b67dcc95922c': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'b534ef17-f251-464a-a83b-b67dcc95922c'}} >Thread-176::INFO::2012-12-04 18:50:14,395::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'b534ef17-f251-464a-a83b-b67dcc95922c': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'b534ef17-f251-464a-a83b-b67dcc95922c'}}} >Thread-176::DEBUG::2012-12-04 18:50:14,395::task::1172::TaskManager.Task::(prepare) Task=`6dfa2d51-04bf-4d79-bda2-307d36f1bf74`::finished: {'allTasksStatus': {'b534ef17-f251-464a-a83b-b67dcc95922c': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'b534ef17-f251-464a-a83b-b67dcc95922c'}}} >Thread-176::DEBUG::2012-12-04 18:50:14,395::task::588::TaskManager.Task::(_updateState) Task=`6dfa2d51-04bf-4d79-bda2-307d36f1bf74`::moving from state preparing -> state finished >Thread-176::DEBUG::2012-12-04 18:50:14,395::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-176::DEBUG::2012-12-04 18:50:14,395::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-176::DEBUG::2012-12-04 18:50:14,395::task::978::TaskManager.Task::(_decref) Task=`6dfa2d51-04bf-4d79-bda2-307d36f1bf74`::ref 0 aborting False >Thread-177::DEBUG::2012-12-04 18:50:14,603::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-177::DEBUG::2012-12-04 18:50:14,603::task::588::TaskManager.Task::(_updateState) Task=`fcda94c2-68bd-4705-8dd1-9e20cd25a19b`::moving from state init -> state preparing >Thread-177::INFO::2012-12-04 18:50:14,603::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='b534ef17-f251-464a-a83b-b67dcc95922c', spUUID=None, options=None) >Thread-177::DEBUG::2012-12-04 18:50:14,603::taskManager::161::TaskManager::(clearTask) Entry. taskID: b534ef17-f251-464a-a83b-b67dcc95922c >Thread-177::DEBUG::2012-12-04 18:50:14,877::taskManager::166::TaskManager::(clearTask) Return. >Thread-177::INFO::2012-12-04 18:50:14,877::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-177::DEBUG::2012-12-04 18:50:14,877::task::1172::TaskManager.Task::(prepare) Task=`fcda94c2-68bd-4705-8dd1-9e20cd25a19b`::finished: None >Thread-177::DEBUG::2012-12-04 18:50:14,878::task::588::TaskManager.Task::(_updateState) Task=`fcda94c2-68bd-4705-8dd1-9e20cd25a19b`::moving from state preparing -> state finished >Thread-177::DEBUG::2012-12-04 18:50:14,878::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-177::DEBUG::2012-12-04 18:50:14,878::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-177::DEBUG::2012-12-04 18:50:14,878::task::978::TaskManager.Task::(_decref) Task=`fcda94c2-68bd-4705-8dd1-9e20cd25a19b`::ref 0 aborting False >Thread-178::DEBUG::2012-12-04 18:50:15,522::task::588::TaskManager.Task::(_updateState) Task=`b301dac7-5534-4203-a537-8b124813fa9e`::moving from state init -> state preparing >Thread-178::INFO::2012-12-04 18:50:15,522::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-178::INFO::2012-12-04 18:50:15,523::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0062689781189', 'lastCheck': 1354639807.514906, 'code': 0, 'valid': True}} >Thread-178::DEBUG::2012-12-04 18:50:15,523::task::1172::TaskManager.Task::(prepare) Task=`b301dac7-5534-4203-a537-8b124813fa9e`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0062689781189', 'lastCheck': 1354639807.514906, 'code': 0, 'valid': True}} >Thread-178::DEBUG::2012-12-04 18:50:15,523::task::588::TaskManager.Task::(_updateState) Task=`b301dac7-5534-4203-a537-8b124813fa9e`::moving from state preparing -> state finished >Thread-178::DEBUG::2012-12-04 18:50:15,523::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-178::DEBUG::2012-12-04 18:50:15,523::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-178::DEBUG::2012-12-04 18:50:15,523::task::978::TaskManager.Task::(_decref) Task=`b301dac7-5534-4203-a537-8b124813fa9e`::ref 0 aborting False >Thread-180::DEBUG::2012-12-04 18:50:15,837::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-180::DEBUG::2012-12-04 18:50:15,837::task::588::TaskManager.Task::(_updateState) Task=`52edc1ac-34e1-46d1-9d5d-2febfd28c566`::moving from state init -> state preparing >Thread-180::INFO::2012-12-04 18:50:15,837::logUtils::37::dispatcher::(wrapper) Run and protect: removeVM(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', vmUUID='282107d0-01d7-4cbc-891a-77eda089aae8', sdUUID=None, options=None) >Thread-180::DEBUG::2012-12-04 18:50:15,838::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`aa7d48a3-ca97-452b-a54b-1cb3708a74b1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-180::DEBUG::2012-12-04 18:50:15,838::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-180::DEBUG::2012-12-04 18:50:15,838::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-180::DEBUG::2012-12-04 18:50:15,838::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`aa7d48a3-ca97-452b-a54b-1cb3708a74b1`::Granted request >Thread-180::DEBUG::2012-12-04 18:50:15,838::task::817::TaskManager.Task::(resourceAcquired) Task=`52edc1ac-34e1-46d1-9d5d-2febfd28c566`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-180::DEBUG::2012-12-04 18:50:15,839::task::978::TaskManager.Task::(_decref) Task=`52edc1ac-34e1-46d1-9d5d-2febfd28c566`::ref 1 aborting False >Thread-180::INFO::2012-12-04 18:50:15,839::sp::1228::Storage.StoragePool::(removeVM) spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb vmUUID=282107d0-01d7-4cbc-891a-77eda089aae8 sdUUID=None >Thread-180::INFO::2012-12-04 18:50:16,434::logUtils::39::dispatcher::(wrapper) Run and protect: removeVM, Return response: None >Thread-180::DEBUG::2012-12-04 18:50:16,434::task::1172::TaskManager.Task::(prepare) Task=`52edc1ac-34e1-46d1-9d5d-2febfd28c566`::finished: None >Thread-180::DEBUG::2012-12-04 18:50:16,434::task::588::TaskManager.Task::(_updateState) Task=`52edc1ac-34e1-46d1-9d5d-2febfd28c566`::moving from state preparing -> state finished >Thread-180::DEBUG::2012-12-04 18:50:16,434::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-180::DEBUG::2012-12-04 18:50:16,435::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-180::DEBUG::2012-12-04 18:50:16,435::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-180::DEBUG::2012-12-04 18:50:16,435::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-180::DEBUG::2012-12-04 18:50:16,435::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-180::DEBUG::2012-12-04 18:50:16,435::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-180::DEBUG::2012-12-04 18:50:16,435::task::978::TaskManager.Task::(_decref) Task=`52edc1ac-34e1-46d1-9d5d-2febfd28c566`::ref 0 aborting False >Thread-181::DEBUG::2012-12-04 18:50:16,557::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-181::DEBUG::2012-12-04 18:50:16,557::task::588::TaskManager.Task::(_updateState) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::moving from state init -> state preparing >Thread-181::INFO::2012-12-04 18:50:16,557::logUtils::37::dispatcher::(wrapper) Run and protect: deleteImage(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', imgUUID='7a7da037-71d4-45f3-99d6-c99721c3ddb3', postZero='false', force='false') >Thread-181::INFO::2012-12-04 18:50:16,558::fileSD::260::Storage.StorageDomain::(validate) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-181::DEBUG::2012-12-04 18:50:19,814::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=49966f75e3c1291add4e8ecc44a5ac665a2d7fe0'] >Thread-181::DEBUG::2012-12-04 18:50:19,815::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3`ReqID=`4e7da82e-e0e1-447c-b391-5c669e0a8e06`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-181::DEBUG::2012-12-04 18:50:19,815::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3' for lock type 'exclusive' >Thread-181::DEBUG::2012-12-04 18:50:19,815::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3' is free. Now locking as 'exclusive' (1 active user) >Thread-181::DEBUG::2012-12-04 18:50:19,816::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3`ReqID=`4e7da82e-e0e1-447c-b391-5c669e0a8e06`::Granted request >Thread-181::DEBUG::2012-12-04 18:50:19,816::task::817::TaskManager.Task::(resourceAcquired) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::_resourcesAcquired: Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3 (exclusive) >Thread-181::DEBUG::2012-12-04 18:50:19,816::task::978::TaskManager.Task::(_decref) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::ref 1 aborting False >Thread-181::DEBUG::2012-12-04 18:50:19,816::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`42811158-4bb1-430d-b538-4de8098ed948`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-181::DEBUG::2012-12-04 18:50:19,816::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'shared' >Thread-181::DEBUG::2012-12-04 18:50:19,817::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'shared' (1 active user) >Thread-181::DEBUG::2012-12-04 18:50:19,817::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`42811158-4bb1-430d-b538-4de8098ed948`::Granted request >Thread-181::DEBUG::2012-12-04 18:50:19,817::task::817::TaskManager.Task::(resourceAcquired) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (shared) >Thread-181::DEBUG::2012-12-04 18:50:19,817::task::978::TaskManager.Task::(_decref) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::ref 1 aborting False >Thread-181::DEBUG::2012-12-04 18:50:19,823::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::DEBUG::2012-12-04 18:50:19,836::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::DEBUG::2012-12-04 18:50:19,850::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::INFO::2012-12-04 18:50:19,857::image::470::Storage.Image::(isLegal) image 7a7da037-71d4-45f3-99d6-c99721c3ddb3 in domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 has vollist ['07a18903-93c1-44fb-a610-60f744cae7b2'] >Thread-181::DEBUG::2012-12-04 18:50:19,858::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::DEBUG::2012-12-04 18:50:19,881::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::DEBUG::2012-12-04 18:50:19,893::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::DEBUG::2012-12-04 18:50:19,904::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::DEBUG::2012-12-04 18:50:19,915::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::INFO::2012-12-04 18:50:19,925::image::470::Storage.Image::(isLegal) image 7a7da037-71d4-45f3-99d6-c99721c3ddb3 in domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 has vollist ['07a18903-93c1-44fb-a610-60f744cae7b2'] >Thread-181::DEBUG::2012-12-04 18:50:19,926::fileVolume::535::Storage.Volume::(validateVolumePath) validate path for 07a18903-93c1-44fb-a610-60f744cae7b2 >Thread-181::INFO::2012-12-04 18:50:19,938::volume::420::Storage.Volume::(setLegality) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 imgUUID=7a7da037-71d4-45f3-99d6-c99721c3ddb3 volUUID = 07a18903-93c1-44fb-a610-60f744cae7b2 legality = ILLEGAL >Thread-181::DEBUG::2012-12-04 18:50:20,076::task::740::TaskManager.Task::(_save) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784.temp >Thread-181::DEBUG::2012-12-04 18:50:20,322::taskManager::66::TaskManager::(scheduleJob) scheduled job deleteImage for task dbbc5edc-4a3f-452b-a086-0b9e22b52784 >Thread-181::INFO::2012-12-04 18:50:20,322::logUtils::39::dispatcher::(wrapper) Run and protect: deleteImage, Return response: None >Thread-181::DEBUG::2012-12-04 18:50:20,322::task::1167::TaskManager.Task::(prepare) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::Prepare: 1 jobs exist, move to acquiring >Thread-181::DEBUG::2012-12-04 18:50:20,322::task::588::TaskManager.Task::(_updateState) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::moving from state preparing -> state acquiring >Thread-181::DEBUG::2012-12-04 18:50:20,347::task::740::TaskManager.Task::(_save) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784.temp >Thread-181::DEBUG::2012-12-04 18:50:20,446::task::588::TaskManager.Task::(_updateState) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::moving from state acquiring -> state queued >Thread-181::DEBUG::2012-12-04 18:50:20,449::task::740::TaskManager.Task::(_save) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784.temp >Thread-181::DEBUG::2012-12-04 18:50:20,526::taskManager::48::TaskManager::(_queueTask) queuing task: dbbc5edc-4a3f-452b-a086-0b9e22b52784 >Thread-181::DEBUG::2012-12-04 18:50:20,526::taskManager::54::TaskManager::(_queueTask) task queued: dbbc5edc-4a3f-452b-a086-0b9e22b52784 >2889e958-982c-4dc6-9d54-1893fe68a8b1::DEBUG::2012-12-04 18:50:20,526::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-181::DEBUG::2012-12-04 18:50:20,527::task::1169::TaskManager.Task::(prepare) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::returning >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,527::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: dbbc5edc-4a3f-452b-a086-0b9e22b52784 running: <bound method Task.commit of <storage.task.Task instance at 0x7f7f140ec908>> with: None >Thread-181::DEBUG::2012-12-04 18:50:20,527::task::978::TaskManager.Task::(_decref) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::ref 0 aborting False >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,527::task::1181::TaskManager.Task::(commit) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::committing task: dbbc5edc-4a3f-452b-a086-0b9e22b52784 >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,528::task::588::TaskManager.Task::(_updateState) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::moving from state queued -> state running >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,530::task::740::TaskManager.Task::(_save) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784.temp >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,588::task::892::TaskManager.Task::(_runJobs) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::Task.run: running job 0: deleteImage: <function <lambda> at 0x7f7f140e5de8> (args: () kwargs: {}) >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,588::task::317::TaskManager.Task::(run) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::Job.run: running deleteImage: <function <lambda> at 0x7f7f140e5de8> (args: () kwargs: {}) callback None >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,588::task::588::TaskManager.Task::(_updateState) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::moving from state running -> state finished >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,593::task::740::TaskManager.Task::(_save) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::_save: orig /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784 temp /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd/master/tasks/dbbc5edc-4a3f-452b-a086-0b9e22b52784.temp >Thread-184::DEBUG::2012-12-04 18:50:20,671::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-184::DEBUG::2012-12-04 18:50:20,672::task::588::TaskManager.Task::(_updateState) Task=`6f72dbba-11f8-477e-ac6c-f9fa0b9d6da5`::moving from state init -> state preparing >Thread-184::INFO::2012-12-04 18:50:20,672::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-184::INFO::2012-12-04 18:50:20,672::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-184::DEBUG::2012-12-04 18:50:20,672::task::1172::TaskManager.Task::(prepare) Task=`6f72dbba-11f8-477e-ac6c-f9fa0b9d6da5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-184::DEBUG::2012-12-04 18:50:20,672::task::588::TaskManager.Task::(_updateState) Task=`6f72dbba-11f8-477e-ac6c-f9fa0b9d6da5`::moving from state preparing -> state finished >Thread-184::DEBUG::2012-12-04 18:50:20,672::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-184::DEBUG::2012-12-04 18:50:20,672::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-184::DEBUG::2012-12-04 18:50:20,673::task::978::TaskManager.Task::(_decref) Task=`6f72dbba-11f8-477e-ac6c-f9fa0b9d6da5`::ref 0 aborting False >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,679::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>, 'Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3': < ResourceRef 'Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3', isValid: 'True' obj: 'None'>} >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,679::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,679::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,679::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,679::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,679::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,680::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3' >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,680::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3' (0 active users) >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,680::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3' is free, finding out if anyone is waiting for it. >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,680::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.7a7da037-71d4-45f3-99d6-c99721c3ddb3', Clearing records. >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,680::task::900::TaskManager.Task::(_runJobs) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::Task.run: exit - success: result True >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,680::task::978::TaskManager.Task::(_decref) Task=`dbbc5edc-4a3f-452b-a086-0b9e22b52784`::ref 0 aborting False >dbbc5edc-4a3f-452b-a086-0b9e22b52784::DEBUG::2012-12-04 18:50:20,681::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-185::DEBUG::2012-12-04 18:50:20,785::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-185::DEBUG::2012-12-04 18:50:20,785::task::588::TaskManager.Task::(_updateState) Task=`b728932c-e3b1-419a-a2a2-5c263d284009`::moving from state init -> state preparing >Thread-185::INFO::2012-12-04 18:50:20,785::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-185::DEBUG::2012-12-04 18:50:20,785::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`a14da678-38bd-48f9-bc31-49786d2cddbe`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-185::DEBUG::2012-12-04 18:50:20,786::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-185::DEBUG::2012-12-04 18:50:20,786::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-185::DEBUG::2012-12-04 18:50:20,786::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`a14da678-38bd-48f9-bc31-49786d2cddbe`::Granted request >Thread-185::DEBUG::2012-12-04 18:50:20,786::task::817::TaskManager.Task::(resourceAcquired) Task=`b728932c-e3b1-419a-a2a2-5c263d284009`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-185::DEBUG::2012-12-04 18:50:20,786::task::978::TaskManager.Task::(_decref) Task=`b728932c-e3b1-419a-a2a2-5c263d284009`::ref 1 aborting False >Thread-185::INFO::2012-12-04 18:50:20,790::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844713951232', 'alerts': [], 'disktotal': '844717096960'}}} >Thread-185::DEBUG::2012-12-04 18:50:20,790::task::1172::TaskManager.Task::(prepare) Task=`b728932c-e3b1-419a-a2a2-5c263d284009`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844713951232', 'alerts': [], 'disktotal': '844717096960'}}} >Thread-185::DEBUG::2012-12-04 18:50:20,790::task::588::TaskManager.Task::(_updateState) Task=`b728932c-e3b1-419a-a2a2-5c263d284009`::moving from state preparing -> state finished >Thread-185::DEBUG::2012-12-04 18:50:20,790::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-185::DEBUG::2012-12-04 18:50:20,790::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-185::DEBUG::2012-12-04 18:50:20,791::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-185::DEBUG::2012-12-04 18:50:20,791::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-185::DEBUG::2012-12-04 18:50:20,791::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-185::DEBUG::2012-12-04 18:50:20,791::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-185::DEBUG::2012-12-04 18:50:20,791::task::978::TaskManager.Task::(_decref) Task=`b728932c-e3b1-419a-a2a2-5c263d284009`::ref 0 aborting False >Thread-188::DEBUG::2012-12-04 18:50:24,452::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-188::DEBUG::2012-12-04 18:50:24,452::task::588::TaskManager.Task::(_updateState) Task=`5bc72794-e4b8-49b3-be09-95031b93e1c7`::moving from state init -> state preparing >Thread-188::INFO::2012-12-04 18:50:24,452::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-188::DEBUG::2012-12-04 18:50:24,452::taskManager::103::TaskManager::(getAllTasksStatuses) Entry. >Thread-188::DEBUG::2012-12-04 18:50:24,452::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: dbbc5edc-4a3f-452b-a086-0b9e22b52784 >Thread-188::DEBUG::2012-12-04 18:50:24,452::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'dbbc5edc-4a3f-452b-a086-0b9e22b52784'} >Thread-188::DEBUG::2012-12-04 18:50:24,452::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {'dbbc5edc-4a3f-452b-a086-0b9e22b52784': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'dbbc5edc-4a3f-452b-a086-0b9e22b52784'}} >Thread-188::INFO::2012-12-04 18:50:24,453::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'dbbc5edc-4a3f-452b-a086-0b9e22b52784': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'dbbc5edc-4a3f-452b-a086-0b9e22b52784'}}} >Thread-188::DEBUG::2012-12-04 18:50:24,453::task::1172::TaskManager.Task::(prepare) Task=`5bc72794-e4b8-49b3-be09-95031b93e1c7`::finished: {'allTasksStatus': {'dbbc5edc-4a3f-452b-a086-0b9e22b52784': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'dbbc5edc-4a3f-452b-a086-0b9e22b52784'}}} >Thread-188::DEBUG::2012-12-04 18:50:24,453::task::588::TaskManager.Task::(_updateState) Task=`5bc72794-e4b8-49b3-be09-95031b93e1c7`::moving from state preparing -> state finished >Thread-188::DEBUG::2012-12-04 18:50:24,453::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-188::DEBUG::2012-12-04 18:50:24,453::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-188::DEBUG::2012-12-04 18:50:24,453::task::978::TaskManager.Task::(_decref) Task=`5bc72794-e4b8-49b3-be09-95031b93e1c7`::ref 0 aborting False >Thread-189::DEBUG::2012-12-04 18:50:24,550::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-189::DEBUG::2012-12-04 18:50:24,550::task::588::TaskManager.Task::(_updateState) Task=`a3307741-f88d-41e3-9356-2dde028ca908`::moving from state init -> state preparing >Thread-189::INFO::2012-12-04 18:50:24,550::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='dbbc5edc-4a3f-452b-a086-0b9e22b52784', spUUID=None, options=None) >Thread-189::DEBUG::2012-12-04 18:50:24,550::taskManager::161::TaskManager::(clearTask) Entry. taskID: dbbc5edc-4a3f-452b-a086-0b9e22b52784 >Thread-189::DEBUG::2012-12-04 18:50:24,558::taskManager::166::TaskManager::(clearTask) Return. >Thread-189::INFO::2012-12-04 18:50:24,558::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-189::DEBUG::2012-12-04 18:50:24,558::task::1172::TaskManager.Task::(prepare) Task=`a3307741-f88d-41e3-9356-2dde028ca908`::finished: None >Thread-189::DEBUG::2012-12-04 18:50:24,558::task::588::TaskManager.Task::(_updateState) Task=`a3307741-f88d-41e3-9356-2dde028ca908`::moving from state preparing -> state finished >Thread-189::DEBUG::2012-12-04 18:50:24,558::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-189::DEBUG::2012-12-04 18:50:24,559::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-189::DEBUG::2012-12-04 18:50:24,559::task::978::TaskManager.Task::(_decref) Task=`a3307741-f88d-41e3-9356-2dde028ca908`::ref 0 aborting False >Thread-190::DEBUG::2012-12-04 18:50:26,203::task::588::TaskManager.Task::(_updateState) Task=`321d6cca-d093-4eda-885e-7984a9cad10f`::moving from state init -> state preparing >Thread-190::INFO::2012-12-04 18:50:26,203::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-190::INFO::2012-12-04 18:50:26,204::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.202747821808', 'lastCheck': 1354639819.822899, 'code': 0, 'valid': True}} >Thread-190::DEBUG::2012-12-04 18:50:26,204::task::1172::TaskManager.Task::(prepare) Task=`321d6cca-d093-4eda-885e-7984a9cad10f`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.202747821808', 'lastCheck': 1354639819.822899, 'code': 0, 'valid': True}} >Thread-190::DEBUG::2012-12-04 18:50:26,204::task::588::TaskManager.Task::(_updateState) Task=`321d6cca-d093-4eda-885e-7984a9cad10f`::moving from state preparing -> state finished >Thread-190::DEBUG::2012-12-04 18:50:26,204::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-190::DEBUG::2012-12-04 18:50:26,204::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-190::DEBUG::2012-12-04 18:50:26,204::task::978::TaskManager.Task::(_decref) Task=`321d6cca-d093-4eda-885e-7984a9cad10f`::ref 0 aborting False >Thread-194::DEBUG::2012-12-04 18:50:30,955::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-194::DEBUG::2012-12-04 18:50:30,955::task::588::TaskManager.Task::(_updateState) Task=`8c2f4ac7-76ec-40af-9938-5f1000edebba`::moving from state init -> state preparing >Thread-194::INFO::2012-12-04 18:50:30,956::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-194::INFO::2012-12-04 18:50:30,956::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-194::DEBUG::2012-12-04 18:50:30,956::task::1172::TaskManager.Task::(prepare) Task=`8c2f4ac7-76ec-40af-9938-5f1000edebba`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-194::DEBUG::2012-12-04 18:50:30,956::task::588::TaskManager.Task::(_updateState) Task=`8c2f4ac7-76ec-40af-9938-5f1000edebba`::moving from state preparing -> state finished >Thread-194::DEBUG::2012-12-04 18:50:30,956::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-194::DEBUG::2012-12-04 18:50:30,956::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-194::DEBUG::2012-12-04 18:50:30,956::task::978::TaskManager.Task::(_decref) Task=`8c2f4ac7-76ec-40af-9938-5f1000edebba`::ref 0 aborting False >Thread-195::DEBUG::2012-12-04 18:50:30,997::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-195::DEBUG::2012-12-04 18:50:30,997::task::588::TaskManager.Task::(_updateState) Task=`8d92c1ee-81d8-400a-9469-30de6d0215f6`::moving from state init -> state preparing >Thread-195::INFO::2012-12-04 18:50:30,997::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-195::DEBUG::2012-12-04 18:50:30,998::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`a030dbe0-15c2-47d9-8128-8f1e2716619f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-195::DEBUG::2012-12-04 18:50:30,998::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-195::DEBUG::2012-12-04 18:50:30,998::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-195::DEBUG::2012-12-04 18:50:30,998::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`a030dbe0-15c2-47d9-8128-8f1e2716619f`::Granted request >Thread-195::DEBUG::2012-12-04 18:50:30,998::task::817::TaskManager.Task::(resourceAcquired) Task=`8d92c1ee-81d8-400a-9469-30de6d0215f6`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-195::DEBUG::2012-12-04 18:50:30,999::task::978::TaskManager.Task::(_decref) Task=`8d92c1ee-81d8-400a-9469-30de6d0215f6`::ref 1 aborting False >Thread-195::INFO::2012-12-04 18:50:31,002::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844775817216', 'alerts': [], 'disktotal': '844780011520'}}} >Thread-195::DEBUG::2012-12-04 18:50:31,002::task::1172::TaskManager.Task::(prepare) Task=`8d92c1ee-81d8-400a-9469-30de6d0215f6`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844775817216', 'alerts': [], 'disktotal': '844780011520'}}} >Thread-195::DEBUG::2012-12-04 18:50:31,002::task::588::TaskManager.Task::(_updateState) Task=`8d92c1ee-81d8-400a-9469-30de6d0215f6`::moving from state preparing -> state finished >Thread-195::DEBUG::2012-12-04 18:50:31,003::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-195::DEBUG::2012-12-04 18:50:31,003::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-195::DEBUG::2012-12-04 18:50:31,003::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-195::DEBUG::2012-12-04 18:50:31,003::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-195::DEBUG::2012-12-04 18:50:31,003::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-195::DEBUG::2012-12-04 18:50:31,003::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-195::DEBUG::2012-12-04 18:50:31,004::task::978::TaskManager.Task::(_decref) Task=`8d92c1ee-81d8-400a-9469-30de6d0215f6`::ref 0 aborting False >Thread-198::DEBUG::2012-12-04 18:50:36,488::task::588::TaskManager.Task::(_updateState) Task=`be6aefe2-fd8b-41be-8126-f1e82039b08b`::moving from state init -> state preparing >Thread-198::INFO::2012-12-04 18:50:36,488::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-198::INFO::2012-12-04 18:50:36,488::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00270700454712', 'lastCheck': 1354639829.83175, 'code': 0, 'valid': True}} >Thread-198::DEBUG::2012-12-04 18:50:36,489::task::1172::TaskManager.Task::(prepare) Task=`be6aefe2-fd8b-41be-8126-f1e82039b08b`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00270700454712', 'lastCheck': 1354639829.83175, 'code': 0, 'valid': True}} >Thread-198::DEBUG::2012-12-04 18:50:36,489::task::588::TaskManager.Task::(_updateState) Task=`be6aefe2-fd8b-41be-8126-f1e82039b08b`::moving from state preparing -> state finished >Thread-198::DEBUG::2012-12-04 18:50:36,489::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-198::DEBUG::2012-12-04 18:50:36,489::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-198::DEBUG::2012-12-04 18:50:36,489::task::978::TaskManager.Task::(_decref) Task=`be6aefe2-fd8b-41be-8126-f1e82039b08b`::ref 0 aborting False >Thread-202::DEBUG::2012-12-04 18:50:41,106::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-202::DEBUG::2012-12-04 18:50:41,106::task::588::TaskManager.Task::(_updateState) Task=`35e412e7-2231-456b-93ee-6d656218edf4`::moving from state init -> state preparing >Thread-202::INFO::2012-12-04 18:50:41,106::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-202::INFO::2012-12-04 18:50:41,106::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-202::DEBUG::2012-12-04 18:50:41,106::task::1172::TaskManager.Task::(prepare) Task=`35e412e7-2231-456b-93ee-6d656218edf4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-202::DEBUG::2012-12-04 18:50:41,106::task::588::TaskManager.Task::(_updateState) Task=`35e412e7-2231-456b-93ee-6d656218edf4`::moving from state preparing -> state finished >Thread-202::DEBUG::2012-12-04 18:50:41,107::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-202::DEBUG::2012-12-04 18:50:41,107::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-202::DEBUG::2012-12-04 18:50:41,107::task::978::TaskManager.Task::(_decref) Task=`35e412e7-2231-456b-93ee-6d656218edf4`::ref 0 aborting False >Thread-203::DEBUG::2012-12-04 18:50:41,157::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-203::DEBUG::2012-12-04 18:50:41,157::task::588::TaskManager.Task::(_updateState) Task=`7e243508-7f82-418f-a1ff-de8997c3f060`::moving from state init -> state preparing >Thread-203::INFO::2012-12-04 18:50:41,157::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-203::DEBUG::2012-12-04 18:50:41,158::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`5586dd96-4e41-409b-b7f1-e61b359e9a3f`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-203::DEBUG::2012-12-04 18:50:41,158::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-203::DEBUG::2012-12-04 18:50:41,158::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-203::DEBUG::2012-12-04 18:50:41,158::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`5586dd96-4e41-409b-b7f1-e61b359e9a3f`::Granted request >Thread-203::DEBUG::2012-12-04 18:50:41,158::task::817::TaskManager.Task::(resourceAcquired) Task=`7e243508-7f82-418f-a1ff-de8997c3f060`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-203::DEBUG::2012-12-04 18:50:41,159::task::978::TaskManager.Task::(_decref) Task=`7e243508-7f82-418f-a1ff-de8997c3f060`::ref 1 aborting False >Thread-203::INFO::2012-12-04 18:50:41,162::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844775817216', 'alerts': [], 'disktotal': '844780011520'}}} >Thread-203::DEBUG::2012-12-04 18:50:41,162::task::1172::TaskManager.Task::(prepare) Task=`7e243508-7f82-418f-a1ff-de8997c3f060`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844775817216', 'alerts': [], 'disktotal': '844780011520'}}} >Thread-203::DEBUG::2012-12-04 18:50:41,162::task::588::TaskManager.Task::(_updateState) Task=`7e243508-7f82-418f-a1ff-de8997c3f060`::moving from state preparing -> state finished >Thread-203::DEBUG::2012-12-04 18:50:41,163::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-203::DEBUG::2012-12-04 18:50:41,163::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-203::DEBUG::2012-12-04 18:50:41,163::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-203::DEBUG::2012-12-04 18:50:41,163::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-203::DEBUG::2012-12-04 18:50:41,163::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-203::DEBUG::2012-12-04 18:50:41,163::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-203::DEBUG::2012-12-04 18:50:41,164::task::978::TaskManager.Task::(_decref) Task=`7e243508-7f82-418f-a1ff-de8997c3f060`::ref 0 aborting False >Thread-206::DEBUG::2012-12-04 18:50:46,751::task::588::TaskManager.Task::(_updateState) Task=`e6aaf6eb-1ae1-46d4-a6d0-79260a655d48`::moving from state init -> state preparing >Thread-206::INFO::2012-12-04 18:50:46,751::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-206::INFO::2012-12-04 18:50:46,751::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00242805480957', 'lastCheck': 1354639839.839742, 'code': 0, 'valid': True}} >Thread-206::DEBUG::2012-12-04 18:50:46,751::task::1172::TaskManager.Task::(prepare) Task=`e6aaf6eb-1ae1-46d4-a6d0-79260a655d48`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.00242805480957', 'lastCheck': 1354639839.839742, 'code': 0, 'valid': True}} >Thread-206::DEBUG::2012-12-04 18:50:46,751::task::588::TaskManager.Task::(_updateState) Task=`e6aaf6eb-1ae1-46d4-a6d0-79260a655d48`::moving from state preparing -> state finished >Thread-206::DEBUG::2012-12-04 18:50:46,751::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-206::DEBUG::2012-12-04 18:50:46,752::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-206::DEBUG::2012-12-04 18:50:46,752::task::978::TaskManager.Task::(_decref) Task=`e6aaf6eb-1ae1-46d4-a6d0-79260a655d48`::ref 0 aborting False >Thread-210::DEBUG::2012-12-04 18:50:51,207::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-210::DEBUG::2012-12-04 18:50:51,208::task::588::TaskManager.Task::(_updateState) Task=`c1828419-a469-4af8-9c36-efb06b738094`::moving from state init -> state preparing >Thread-210::INFO::2012-12-04 18:50:51,208::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-210::INFO::2012-12-04 18:50:51,208::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-210::DEBUG::2012-12-04 18:50:51,208::task::1172::TaskManager.Task::(prepare) Task=`c1828419-a469-4af8-9c36-efb06b738094`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-210::DEBUG::2012-12-04 18:50:51,208::task::588::TaskManager.Task::(_updateState) Task=`c1828419-a469-4af8-9c36-efb06b738094`::moving from state preparing -> state finished >Thread-210::DEBUG::2012-12-04 18:50:51,208::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-210::DEBUG::2012-12-04 18:50:51,208::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-210::DEBUG::2012-12-04 18:50:51,209::task::978::TaskManager.Task::(_decref) Task=`c1828419-a469-4af8-9c36-efb06b738094`::ref 0 aborting False >Thread-211::DEBUG::2012-12-04 18:50:51,238::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-211::DEBUG::2012-12-04 18:50:51,238::task::588::TaskManager.Task::(_updateState) Task=`09c04afe-b045-485b-a298-e7d398998c66`::moving from state init -> state preparing >Thread-211::INFO::2012-12-04 18:50:51,239::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-211::DEBUG::2012-12-04 18:50:51,239::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`eb8b8965-3478-4fdd-8c70-c4410e2f04a3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-211::DEBUG::2012-12-04 18:50:51,239::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-211::DEBUG::2012-12-04 18:50:51,239::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-211::DEBUG::2012-12-04 18:50:51,239::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`eb8b8965-3478-4fdd-8c70-c4410e2f04a3`::Granted request >Thread-211::DEBUG::2012-12-04 18:50:51,240::task::817::TaskManager.Task::(resourceAcquired) Task=`09c04afe-b045-485b-a298-e7d398998c66`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-211::DEBUG::2012-12-04 18:50:51,240::task::978::TaskManager.Task::(_decref) Task=`09c04afe-b045-485b-a298-e7d398998c66`::ref 1 aborting False >Thread-211::INFO::2012-12-04 18:50:51,243::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844669911040', 'alerts': [], 'disktotal': '844672008192'}}} >Thread-211::DEBUG::2012-12-04 18:50:51,243::task::1172::TaskManager.Task::(prepare) Task=`09c04afe-b045-485b-a298-e7d398998c66`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844669911040', 'alerts': [], 'disktotal': '844672008192'}}} >Thread-211::DEBUG::2012-12-04 18:50:51,244::task::588::TaskManager.Task::(_updateState) Task=`09c04afe-b045-485b-a298-e7d398998c66`::moving from state preparing -> state finished >Thread-211::DEBUG::2012-12-04 18:50:51,244::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-211::DEBUG::2012-12-04 18:50:51,244::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-211::DEBUG::2012-12-04 18:50:51,244::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-211::DEBUG::2012-12-04 18:50:51,244::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-211::DEBUG::2012-12-04 18:50:51,244::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-211::DEBUG::2012-12-04 18:50:51,244::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-211::DEBUG::2012-12-04 18:50:51,245::task::978::TaskManager.Task::(_decref) Task=`09c04afe-b045-485b-a298-e7d398998c66`::ref 0 aborting False >Thread-214::DEBUG::2012-12-04 18:50:57,063::task::588::TaskManager.Task::(_updateState) Task=`9535bb52-5ef9-4ed1-b604-c93d9b051ff0`::moving from state init -> state preparing >Thread-214::INFO::2012-12-04 18:50:57,063::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-214::INFO::2012-12-04 18:50:57,063::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0175099372864', 'lastCheck': 1354639849.872381, 'code': 0, 'valid': True}} >Thread-214::DEBUG::2012-12-04 18:50:57,064::task::1172::TaskManager.Task::(prepare) Task=`9535bb52-5ef9-4ed1-b604-c93d9b051ff0`::finished: {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'delay': '0.0175099372864', 'lastCheck': 1354639849.872381, 'code': 0, 'valid': True}} >Thread-214::DEBUG::2012-12-04 18:50:57,064::task::588::TaskManager.Task::(_updateState) Task=`9535bb52-5ef9-4ed1-b604-c93d9b051ff0`::moving from state preparing -> state finished >Thread-214::DEBUG::2012-12-04 18:50:57,064::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-214::DEBUG::2012-12-04 18:50:57,064::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-214::DEBUG::2012-12-04 18:50:57,064::task::978::TaskManager.Task::(_decref) Task=`9535bb52-5ef9-4ed1-b604-c93d9b051ff0`::ref 0 aborting False >Thread-218::DEBUG::2012-12-04 18:51:01,299::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-218::DEBUG::2012-12-04 18:51:01,299::task::588::TaskManager.Task::(_updateState) Task=`017c66f9-467e-4b70-815f-d8cb804fd9b5`::moving from state init -> state preparing >Thread-218::INFO::2012-12-04 18:51:01,300::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-218::INFO::2012-12-04 18:51:01,300::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-218::DEBUG::2012-12-04 18:51:01,300::task::1172::TaskManager.Task::(prepare) Task=`017c66f9-467e-4b70-815f-d8cb804fd9b5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-218::DEBUG::2012-12-04 18:51:01,300::task::588::TaskManager.Task::(_updateState) Task=`017c66f9-467e-4b70-815f-d8cb804fd9b5`::moving from state preparing -> state finished >Thread-218::DEBUG::2012-12-04 18:51:01,300::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-218::DEBUG::2012-12-04 18:51:01,300::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-218::DEBUG::2012-12-04 18:51:01,300::task::978::TaskManager.Task::(_decref) Task=`017c66f9-467e-4b70-815f-d8cb804fd9b5`::ref 0 aborting False >Thread-219::DEBUG::2012-12-04 18:51:01,386::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-219::DEBUG::2012-12-04 18:51:01,386::task::588::TaskManager.Task::(_updateState) Task=`eb718835-2416-42fa-bd5f-f9bb757ea184`::moving from state init -> state preparing >Thread-219::INFO::2012-12-04 18:51:01,386::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-219::DEBUG::2012-12-04 18:51:01,387::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`6393b1a2-f983-4986-adca-6eb236eb10bf`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-219::DEBUG::2012-12-04 18:51:01,387::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'shared' >Thread-219::DEBUG::2012-12-04 18:51:01,387::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'shared' (1 active user) >Thread-219::DEBUG::2012-12-04 18:51:01,387::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`6393b1a2-f983-4986-adca-6eb236eb10bf`::Granted request >Thread-219::DEBUG::2012-12-04 18:51:01,387::task::817::TaskManager.Task::(resourceAcquired) Task=`eb718835-2416-42fa-bd5f-f9bb757ea184`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (shared) >Thread-219::DEBUG::2012-12-04 18:51:01,387::task::978::TaskManager.Task::(_decref) Task=`eb718835-2416-42fa-bd5f-f9bb757ea184`::ref 1 aborting False >Thread-219::INFO::2012-12-04 18:51:01,391::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844762185728', 'alerts': [], 'disktotal': '844764282880'}}} >Thread-219::DEBUG::2012-12-04 18:51:01,391::task::1172::TaskManager.Task::(prepare) Task=`eb718835-2416-42fa-bd5f-f9bb757ea184`::finished: {'info': {'spm_id': 1, 'master_uuid': '8dc966ae-7319-4db8-8fab-33f01a0512b6', 'name': 'DefaultRest', 'version': '3', 'domains': '8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'8dc966ae-7319-4db8-8fab-33f01a0512b6': {'status': 'Active', 'diskfree': '844762185728', 'alerts': [], 'disktotal': '844764282880'}}} >Thread-219::DEBUG::2012-12-04 18:51:01,391::task::588::TaskManager.Task::(_updateState) Task=`eb718835-2416-42fa-bd5f-f9bb757ea184`::moving from state preparing -> state finished >Thread-219::DEBUG::2012-12-04 18:51:01,391::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-219::DEBUG::2012-12-04 18:51:01,392::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-219::DEBUG::2012-12-04 18:51:01,392::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-219::DEBUG::2012-12-04 18:51:01,392::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-219::DEBUG::2012-12-04 18:51:01,392::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-219::DEBUG::2012-12-04 18:51:01,392::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-219::DEBUG::2012-12-04 18:51:01,392::task::978::TaskManager.Task::(_decref) Task=`eb718835-2416-42fa-bd5f-f9bb757ea184`::ref 0 aborting False >Thread-220::DEBUG::2012-12-04 18:51:02,263::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-220::DEBUG::2012-12-04 18:51:02,263::task::588::TaskManager.Task::(_updateState) Task=`e1e2b24a-5263-46be-a8dc-d27b5c7873ce`::moving from state init -> state preparing >Thread-220::INFO::2012-12-04 18:51:02,263::logUtils::37::dispatcher::(wrapper) Run and protect: deactivateStorageDomain(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', msdUUID='00000000-0000-0000-0000-000000000000', masterVersion=1, options=None) >Thread-220::DEBUG::2012-12-04 18:51:02,264::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`ee989673-bdd1-48be-8007-72572b89f9db`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-220::DEBUG::2012-12-04 18:51:02,264::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-220::DEBUG::2012-12-04 18:51:02,264::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-220::DEBUG::2012-12-04 18:51:02,264::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`ee989673-bdd1-48be-8007-72572b89f9db`::Granted request >Thread-220::DEBUG::2012-12-04 18:51:02,264::task::817::TaskManager.Task::(resourceAcquired) Task=`e1e2b24a-5263-46be-a8dc-d27b5c7873ce`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (exclusive) >Thread-220::DEBUG::2012-12-04 18:51:02,265::task::978::TaskManager.Task::(_decref) Task=`e1e2b24a-5263-46be-a8dc-d27b5c7873ce`::ref 1 aborting False >Thread-220::DEBUG::2012-12-04 18:51:02,265::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`15f15109-27c4-4feb-a2d2-a0919a654056`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-220::DEBUG::2012-12-04 18:51:02,265::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'exclusive' >Thread-220::DEBUG::2012-12-04 18:51:02,265::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'exclusive' (1 active user) >Thread-220::DEBUG::2012-12-04 18:51:02,265::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`15f15109-27c4-4feb-a2d2-a0919a654056`::Granted request >Thread-220::DEBUG::2012-12-04 18:51:02,266::task::817::TaskManager.Task::(resourceAcquired) Task=`e1e2b24a-5263-46be-a8dc-d27b5c7873ce`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (exclusive) >Thread-220::DEBUG::2012-12-04 18:51:02,266::task::978::TaskManager.Task::(_decref) Task=`e1e2b24a-5263-46be-a8dc-d27b5c7873ce`::ref 1 aborting False >Thread-220::INFO::2012-12-04 18:51:02,266::sp::1022::Storage.StoragePool::(deactivateSD) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb newMsdUUID=00000000-0000-0000-0000-000000000000 >Thread-220::INFO::2012-12-04 18:51:02,266::fileSD::260::Storage.StorageDomain::(validate) sdUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-220::DEBUG::2012-12-04 18:51:02,268::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=49966f75e3c1291add4e8ecc44a5ac665a2d7fe0'] >Thread-220::INFO::2012-12-04 18:51:02,269::sp::1049::Storage.StoragePool::(deactivateSD) Silently ignoring the deactivation request for the master domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-220::INFO::2012-12-04 18:51:02,269::logUtils::39::dispatcher::(wrapper) Run and protect: deactivateStorageDomain, Return response: None >Thread-220::DEBUG::2012-12-04 18:51:02,269::task::1172::TaskManager.Task::(prepare) Task=`e1e2b24a-5263-46be-a8dc-d27b5c7873ce`::finished: None >Thread-220::DEBUG::2012-12-04 18:51:02,269::task::588::TaskManager.Task::(_updateState) Task=`e1e2b24a-5263-46be-a8dc-d27b5c7873ce`::moving from state preparing -> state finished >Thread-220::DEBUG::2012-12-04 18:51:02,269::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>, 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-220::DEBUG::2012-12-04 18:51:02,269::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-220::DEBUG::2012-12-04 18:51:02,270::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-220::DEBUG::2012-12-04 18:51:02,270::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-220::DEBUG::2012-12-04 18:51:02,270::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-220::DEBUG::2012-12-04 18:51:02,270::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-220::DEBUG::2012-12-04 18:51:02,270::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-220::DEBUG::2012-12-04 18:51:02,270::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-220::DEBUG::2012-12-04 18:51:02,271::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-220::DEBUG::2012-12-04 18:51:02,271::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-220::DEBUG::2012-12-04 18:51:02,271::task::978::TaskManager.Task::(_decref) Task=`e1e2b24a-5263-46be-a8dc-d27b5c7873ce`::ref 0 aborting False >Thread-221::DEBUG::2012-12-04 18:51:02,302::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-221::DEBUG::2012-12-04 18:51:02,302::task::588::TaskManager.Task::(_updateState) Task=`60d2a500-91a1-4b2b-9828-ddf7bad5cf56`::moving from state init -> state preparing >Thread-221::INFO::2012-12-04 18:51:02,302::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-221::DEBUG::2012-12-04 18:51:02,302::taskManager::103::TaskManager::(getAllTasksStatuses) Entry. >Thread-221::DEBUG::2012-12-04 18:51:02,303::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {} >Thread-221::INFO::2012-12-04 18:51:02,303::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-221::DEBUG::2012-12-04 18:51:02,303::task::1172::TaskManager.Task::(prepare) Task=`60d2a500-91a1-4b2b-9828-ddf7bad5cf56`::finished: {'allTasksStatus': {}} >Thread-221::DEBUG::2012-12-04 18:51:02,303::task::588::TaskManager.Task::(_updateState) Task=`60d2a500-91a1-4b2b-9828-ddf7bad5cf56`::moving from state preparing -> state finished >Thread-221::DEBUG::2012-12-04 18:51:02,303::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-221::DEBUG::2012-12-04 18:51:02,303::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-221::DEBUG::2012-12-04 18:51:02,303::task::978::TaskManager.Task::(_decref) Task=`60d2a500-91a1-4b2b-9828-ddf7bad5cf56`::ref 0 aborting False >Thread-222::DEBUG::2012-12-04 18:51:02,352::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-222::DEBUG::2012-12-04 18:51:02,352::task::588::TaskManager.Task::(_updateState) Task=`0a77159d-6fac-4c74-87b7-7639b0d11af7`::moving from state init -> state preparing >Thread-222::INFO::2012-12-04 18:51:02,352::logUtils::37::dispatcher::(wrapper) Run and protect: spmStop(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-222::DEBUG::2012-12-04 18:51:02,352::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`70ecc468-f467-438a-8310-d40f4442e6c4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-222::DEBUG::2012-12-04 18:51:02,353::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-222::DEBUG::2012-12-04 18:51:02,353::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-222::DEBUG::2012-12-04 18:51:02,353::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`70ecc468-f467-438a-8310-d40f4442e6c4`::Granted request >Thread-222::DEBUG::2012-12-04 18:51:02,353::task::817::TaskManager.Task::(resourceAcquired) Task=`0a77159d-6fac-4c74-87b7-7639b0d11af7`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (exclusive) >Thread-222::DEBUG::2012-12-04 18:51:02,353::task::978::TaskManager.Task::(_decref) Task=`0a77159d-6fac-4c74-87b7-7639b0d11af7`::ref 1 aborting False >Thread-222::DEBUG::2012-12-04 18:51:02,354::sp::319::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process >Thread-222::DEBUG::2012-12-04 18:51:02,354::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`4d81c739-0787-48f7-938f-96d643f52329`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-222::DEBUG::2012-12-04 18:51:02,354::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-222::DEBUG::2012-12-04 18:51:02,354::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-222::DEBUG::2012-12-04 18:51:02,354::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`4d81c739-0787-48f7-938f-96d643f52329`::Granted request >Thread-222::DEBUG::2012-12-04 18:51:02,355::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-222::DEBUG::2012-12-04 18:51:02,355::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-222::DEBUG::2012-12-04 18:51:02,355::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-222::DEBUG::2012-12-04 18:51:02,355::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-222::DEBUG::2012-12-04 18:51:02,355::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-222::DEBUG::2012-12-04 18:51:02,356::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-222::DEBUG::2012-12-04 18:51:02,356::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=562119f06d0985abd7d407725393c194ad740af6'] >Thread-222::DEBUG::2012-12-04 18:51:02,359::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-222::INFO::2012-12-04 18:51:02,359::safelease::218::SANLock::(release) Releasing cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 1) >Thread-222::DEBUG::2012-12-04 18:51:02,362::safelease::229::SANLock::(release) Cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully released (id: 1) >Thread-222::INFO::2012-12-04 18:51:02,362::logUtils::39::dispatcher::(wrapper) Run and protect: spmStop, Return response: None >Thread-222::DEBUG::2012-12-04 18:51:02,362::task::1172::TaskManager.Task::(prepare) Task=`0a77159d-6fac-4c74-87b7-7639b0d11af7`::finished: None >Thread-222::DEBUG::2012-12-04 18:51:02,362::task::588::TaskManager.Task::(_updateState) Task=`0a77159d-6fac-4c74-87b7-7639b0d11af7`::moving from state preparing -> state finished >Thread-222::DEBUG::2012-12-04 18:51:02,363::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-222::DEBUG::2012-12-04 18:51:02,363::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-222::DEBUG::2012-12-04 18:51:02,363::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-222::DEBUG::2012-12-04 18:51:02,363::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-222::DEBUG::2012-12-04 18:51:02,363::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-222::DEBUG::2012-12-04 18:51:02,363::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-222::DEBUG::2012-12-04 18:51:02,363::task::978::TaskManager.Task::(_decref) Task=`0a77159d-6fac-4c74-87b7-7639b0d11af7`::ref 0 aborting False >Thread-223::DEBUG::2012-12-04 18:51:02,418::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-223::DEBUG::2012-12-04 18:51:02,418::task::588::TaskManager.Task::(_updateState) Task=`9e9ed0a1-d771-478a-8883-985bd285a3d8`::moving from state init -> state preparing >Thread-223::INFO::2012-12-04 18:51:02,418::logUtils::37::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', hostID=1, scsiKey='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', remove=False, options=None) >Thread-223::DEBUG::2012-12-04 18:51:02,419::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`18b839e8-e161-4281-a150-189a6dc59ad2`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-223::DEBUG::2012-12-04 18:51:02,419::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-223::DEBUG::2012-12-04 18:51:02,419::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-223::DEBUG::2012-12-04 18:51:02,419::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`18b839e8-e161-4281-a150-189a6dc59ad2`::Granted request >Thread-223::DEBUG::2012-12-04 18:51:02,419::task::817::TaskManager.Task::(resourceAcquired) Task=`9e9ed0a1-d771-478a-8883-985bd285a3d8`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (exclusive) >Thread-223::DEBUG::2012-12-04 18:51:02,419::task::978::TaskManager.Task::(_decref) Task=`9e9ed0a1-d771-478a-8883-985bd285a3d8`::ref 1 aborting False >Thread-223::INFO::2012-12-04 18:51:02,420::sp::668::Storage.StoragePool::(disconnect) Disconnect from the storage pool ac5d6452-d8fe-4d69-8d98-d99cd5a494bb >Thread-223::INFO::2012-12-04 18:51:02,420::safelease::171::SANLock::(releaseHostId) Releasing host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 1) >Thread-223::DEBUG::2012-12-04 18:51:05,421::safelease::180::SANLock::(releaseHostId) Host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 released successfully (id: None) >Thread-223::INFO::2012-12-04 18:51:05,421::logUtils::39::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: True >Thread-223::DEBUG::2012-12-04 18:51:05,421::task::1172::TaskManager.Task::(prepare) Task=`9e9ed0a1-d771-478a-8883-985bd285a3d8`::finished: True >Thread-223::DEBUG::2012-12-04 18:51:05,421::task::588::TaskManager.Task::(_updateState) Task=`9e9ed0a1-d771-478a-8883-985bd285a3d8`::moving from state preparing -> state finished >Thread-223::DEBUG::2012-12-04 18:51:05,421::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-223::DEBUG::2012-12-04 18:51:05,422::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-223::DEBUG::2012-12-04 18:51:05,422::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-223::DEBUG::2012-12-04 18:51:05,422::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-223::DEBUG::2012-12-04 18:51:05,422::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-223::DEBUG::2012-12-04 18:51:05,422::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-223::DEBUG::2012-12-04 18:51:05,422::task::978::TaskManager.Task::(_decref) Task=`9e9ed0a1-d771-478a-8883-985bd285a3d8`::ref 0 aborting False >Thread-226::DEBUG::2012-12-04 18:51:05,462::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-226::DEBUG::2012-12-04 18:51:05,462::task::588::TaskManager.Task::(_updateState) Task=`625d1a27-cabe-47ea-b16e-8619b8e27b49`::moving from state init -> state preparing >Thread-226::INFO::2012-12-04 18:51:05,462::logUtils::37::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-226::DEBUG::2012-12-04 18:51:05,463::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604' (cwd None) >Thread-226::DEBUG::2012-12-04 18:51:05,497::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-226::DEBUG::2012-12-04 18:51:05,497::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-226::DEBUG::2012-12-04 18:51:05,498::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-226::DEBUG::2012-12-04 18:51:05,498::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-226::DEBUG::2012-12-04 18:51:05,498::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-226::DEBUG::2012-12-04 18:51:05,507::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-226::DEBUG::2012-12-04 18:51:05,507::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-226::DEBUG::2012-12-04 18:51:07,341::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-226::DEBUG::2012-12-04 18:51:07,404::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-226::DEBUG::2012-12-04 18:51:07,404::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-226::DEBUG::2012-12-04 18:51:07,404::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-226::DEBUG::2012-12-04 18:51:07,404::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-226::DEBUG::2012-12-04 18:51:07,405::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-226::DEBUG::2012-12-04 18:51:07,405::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-226::DEBUG::2012-12-04 18:51:07,405::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-226::DEBUG::2012-12-04 18:51:07,405::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-226::INFO::2012-12-04 18:51:07,405::logUtils::39::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-226::DEBUG::2012-12-04 18:51:07,406::task::1172::TaskManager.Task::(prepare) Task=`625d1a27-cabe-47ea-b16e-8619b8e27b49`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-226::DEBUG::2012-12-04 18:51:07,406::task::588::TaskManager.Task::(_updateState) Task=`625d1a27-cabe-47ea-b16e-8619b8e27b49`::moving from state preparing -> state finished >Thread-227::DEBUG::2012-12-04 18:51:07,407::task::588::TaskManager.Task::(_updateState) Task=`2f142bec-c313-4ffb-b477-50b1500161eb`::moving from state init -> state preparing >Thread-226::DEBUG::2012-12-04 18:51:07,407::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-227::INFO::2012-12-04 18:51:07,407::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-226::DEBUG::2012-12-04 18:51:07,408::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-227::INFO::2012-12-04 18:51:07,408::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-226::DEBUG::2012-12-04 18:51:07,408::task::978::TaskManager.Task::(_decref) Task=`625d1a27-cabe-47ea-b16e-8619b8e27b49`::ref 0 aborting False >Thread-227::DEBUG::2012-12-04 18:51:07,408::task::1172::TaskManager.Task::(prepare) Task=`2f142bec-c313-4ffb-b477-50b1500161eb`::finished: {} >Thread-227::DEBUG::2012-12-04 18:51:07,409::task::588::TaskManager.Task::(_updateState) Task=`2f142bec-c313-4ffb-b477-50b1500161eb`::moving from state preparing -> state finished >Thread-227::DEBUG::2012-12-04 18:51:07,409::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-227::DEBUG::2012-12-04 18:51:07,409::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-227::DEBUG::2012-12-04 18:51:07,409::task::978::TaskManager.Task::(_decref) Task=`2f142bec-c313-4ffb-b477-50b1500161eb`::ref 0 aborting False >Thread-229::DEBUG::2012-12-04 18:51:08,189::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-229::DEBUG::2012-12-04 18:51:08,189::task::588::TaskManager.Task::(_updateState) Task=`1b3c7c0b-3b7b-4e41-b18b-594a2a7834a7`::moving from state init -> state preparing >Thread-229::INFO::2012-12-04 18:51:08,189::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-229::INFO::2012-12-04 18:51:08,190::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-229::DEBUG::2012-12-04 18:51:08,190::task::1172::TaskManager.Task::(prepare) Task=`1b3c7c0b-3b7b-4e41-b18b-594a2a7834a7`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-229::DEBUG::2012-12-04 18:51:08,190::task::588::TaskManager.Task::(_updateState) Task=`1b3c7c0b-3b7b-4e41-b18b-594a2a7834a7`::moving from state preparing -> state finished >Thread-229::DEBUG::2012-12-04 18:51:08,190::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-229::DEBUG::2012-12-04 18:51:08,190::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-229::DEBUG::2012-12-04 18:51:08,190::task::978::TaskManager.Task::(_decref) Task=`1b3c7c0b-3b7b-4e41-b18b-594a2a7834a7`::ref 0 aborting False >Thread-230::DEBUG::2012-12-04 18:51:08,228::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-230::DEBUG::2012-12-04 18:51:08,228::task::588::TaskManager.Task::(_updateState) Task=`26bffc41-f93c-4c76-9036-49a3c389eb21`::moving from state init -> state preparing >Thread-230::INFO::2012-12-04 18:51:08,228::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-230::DEBUG::2012-12-04 18:51:08,232::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604 /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604' (cwd None) >Thread-230::DEBUG::2012-12-04 18:51:08,360::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-230::DEBUG::2012-12-04 18:51:08,360::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-230::DEBUG::2012-12-04 18:51:08,361::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-230::DEBUG::2012-12-04 18:51:08,361::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-230::DEBUG::2012-12-04 18:51:08,361::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-230::DEBUG::2012-12-04 18:51:08,361::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-230::INFO::2012-12-04 18:51:08,361::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-230::DEBUG::2012-12-04 18:51:08,362::task::1172::TaskManager.Task::(prepare) Task=`26bffc41-f93c-4c76-9036-49a3c389eb21`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-230::DEBUG::2012-12-04 18:51:08,362::task::588::TaskManager.Task::(_updateState) Task=`26bffc41-f93c-4c76-9036-49a3c389eb21`::moving from state preparing -> state finished >Thread-230::DEBUG::2012-12-04 18:51:08,362::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-230::DEBUG::2012-12-04 18:51:08,362::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-230::DEBUG::2012-12-04 18:51:08,362::task::978::TaskManager.Task::(_decref) Task=`26bffc41-f93c-4c76-9036-49a3c389eb21`::ref 0 aborting False >Thread-231::DEBUG::2012-12-04 18:51:08,393::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-231::DEBUG::2012-12-04 18:51:08,393::task::588::TaskManager.Task::(_updateState) Task=`09134d34-b763-4160-b940-07b544b66e2e`::moving from state init -> state preparing >Thread-231::INFO::2012-12-04 18:51:08,393::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', hostID=1, scsiKey='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', msdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', masterVersion=1, options=None) >Thread-231::DEBUG::2012-12-04 18:51:08,394::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`90784a53-a59f-4a55-bbf4-5c0b4ccc2fc8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-231::DEBUG::2012-12-04 18:51:08,394::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-231::DEBUG::2012-12-04 18:51:08,394::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-231::DEBUG::2012-12-04 18:51:08,394::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`90784a53-a59f-4a55-bbf4-5c0b4ccc2fc8`::Granted request >Thread-231::INFO::2012-12-04 18:51:08,395::sp::625::Storage.StoragePool::(connect) Connect host #1 to the storage pool ac5d6452-d8fe-4d69-8d98-d99cd5a494bb with master domain: 8dc966ae-7319-4db8-8fab-33f01a0512b6 (ver = 1) >Thread-231::DEBUG::2012-12-04 18:51:08,395::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:08,395::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:08,396::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:08,396::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:08,396::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:08,396::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:08,396::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-231::DEBUG::2012-12-04 18:51:08,396::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-231::DEBUG::2012-12-04 18:51:08,396::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-231::DEBUG::2012-12-04 18:51:08,397::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-231::DEBUG::2012-12-04 18:51:08,397::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-231::DEBUG::2012-12-04 18:51:08,406::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-231::DEBUG::2012-12-04 18:51:08,406::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-231::DEBUG::2012-12-04 18:51:10,240::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-231::DEBUG::2012-12-04 18:51:10,272::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-231::DEBUG::2012-12-04 18:51:10,272::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,272::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,272::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,273::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,273::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,273::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,273::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-231::DEBUG::2012-12-04 18:51:10,273::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,274::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 8dc966ae-7319-4db8-8fab-33f01a0512b6' (cwd None) >Thread-231::DEBUG::2012-12-04 18:51:10,293::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found\n'; <rc> = 5 >Thread-231::WARNING::2012-12-04 18:51:10,294::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found'] >Thread-231::DEBUG::2012-12-04 18:51:10,295::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,301::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-231::DEBUG::2012-12-04 18:51:10,302::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-231::DEBUG::2012-12-04 18:51:10,306::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=562119f06d0985abd7d407725393c194ad740af6'] >Thread-231::DEBUG::2012-12-04 18:51:10,308::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-231::WARNING::2012-12-04 18:51:10,308::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS already registered >Thread-231::WARNING::2012-12-04 18:51:10,309::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS already registered >Thread-231::DEBUG::2012-12-04 18:51:10,309::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 verified, version 1 >Thread-231::DEBUG::2012-12-04 18:51:10,309::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-231::DEBUG::2012-12-04 18:51:10,309::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-231::DEBUG::2012-12-04 18:51:10,312::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=562119f06d0985abd7d407725393c194ad740af6'] >Thread-231::DEBUG::2012-12-04 18:51:10,312::sp::1495::Storage.StoragePool::(updateMonitoringThreads) sp `ac5d6452-d8fe-4d69-8d98-d99cd5a494bb` started monitoring domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` >Thread-231::DEBUG::2012-12-04 18:51:10,312::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-231::DEBUG::2012-12-04 18:51:10,315::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-231::DEBUG::2012-12-04 18:51:10,315::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-231::DEBUG::2012-12-04 18:51:10,333::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-231::DEBUG::2012-12-04 18:51:10,333::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-231::INFO::2012-12-04 18:51:10,335::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 to /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-231::INFO::2012-12-04 18:51:10,335::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 to /rhev/data-center/ac5d6452-d8fe-4d69-8d98-d99cd5a494bb/mastersd >Thread-231::INFO::2012-12-04 18:51:10,336::safelease::156::SANLock::(acquireHostId) Acquiring host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 1) >Thread-239::DEBUG::2012-12-04 18:51:17,760::task::588::TaskManager.Task::(_updateState) Task=`d60288a4-3be1-4728-a547-df37683cb702`::moving from state init -> state preparing >Thread-239::INFO::2012-12-04 18:51:17,761::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-239::INFO::2012-12-04 18:51:17,761::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-239::DEBUG::2012-12-04 18:51:17,761::task::1172::TaskManager.Task::(prepare) Task=`d60288a4-3be1-4728-a547-df37683cb702`::finished: {} >Thread-239::DEBUG::2012-12-04 18:51:17,761::task::588::TaskManager.Task::(_updateState) Task=`d60288a4-3be1-4728-a547-df37683cb702`::moving from state preparing -> state finished >Thread-239::DEBUG::2012-12-04 18:51:17,761::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-239::DEBUG::2012-12-04 18:51:17,761::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-239::DEBUG::2012-12-04 18:51:17,761::task::978::TaskManager.Task::(_decref) Task=`d60288a4-3be1-4728-a547-df37683cb702`::ref 0 aborting False >Thread-245::DEBUG::2012-12-04 18:51:28,032::task::588::TaskManager.Task::(_updateState) Task=`95e53f6a-8f87-4ae6-b39c-d64be7f4fcc6`::moving from state init -> state preparing >Thread-245::INFO::2012-12-04 18:51:28,033::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-245::INFO::2012-12-04 18:51:28,033::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-245::DEBUG::2012-12-04 18:51:28,033::task::1172::TaskManager.Task::(prepare) Task=`95e53f6a-8f87-4ae6-b39c-d64be7f4fcc6`::finished: {} >Thread-245::DEBUG::2012-12-04 18:51:28,033::task::588::TaskManager.Task::(_updateState) Task=`95e53f6a-8f87-4ae6-b39c-d64be7f4fcc6`::moving from state preparing -> state finished >Thread-245::DEBUG::2012-12-04 18:51:28,033::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-245::DEBUG::2012-12-04 18:51:28,033::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-245::DEBUG::2012-12-04 18:51:28,033::task::978::TaskManager.Task::(_decref) Task=`95e53f6a-8f87-4ae6-b39c-d64be7f4fcc6`::ref 0 aborting False >Thread-231::DEBUG::2012-12-04 18:51:31,339::safelease::166::SANLock::(acquireHostId) Host id for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully acquired (id: 1) >Thread-231::DEBUG::2012-12-04 18:51:31,339::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-231::DEBUG::2012-12-04 18:51:31,339::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-231::DEBUG::2012-12-04 18:51:31,339::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-231::DEBUG::2012-12-04 18:51:31,339::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-231::INFO::2012-12-04 18:51:31,340::logUtils::39::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-231::DEBUG::2012-12-04 18:51:31,340::task::1172::TaskManager.Task::(prepare) Task=`09134d34-b763-4160-b940-07b544b66e2e`::finished: True >Thread-231::DEBUG::2012-12-04 18:51:31,340::task::588::TaskManager.Task::(_updateState) Task=`09134d34-b763-4160-b940-07b544b66e2e`::moving from state preparing -> state finished >Thread-231::DEBUG::2012-12-04 18:51:31,340::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-231::DEBUG::2012-12-04 18:51:31,340::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-231::DEBUG::2012-12-04 18:51:31,340::task::978::TaskManager.Task::(_decref) Task=`09134d34-b763-4160-b940-07b544b66e2e`::ref 0 aborting False >Thread-248::DEBUG::2012-12-04 18:51:31,502::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-248::DEBUG::2012-12-04 18:51:31,502::task::588::TaskManager.Task::(_updateState) Task=`0a3a9082-69c3-4671-850e-c610f30d47a8`::moving from state init -> state preparing >Thread-248::INFO::2012-12-04 18:51:31,502::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-248::INFO::2012-12-04 18:51:31,502::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 0}} >Thread-248::DEBUG::2012-12-04 18:51:31,502::task::1172::TaskManager.Task::(prepare) Task=`0a3a9082-69c3-4671-850e-c610f30d47a8`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 0}} >Thread-248::DEBUG::2012-12-04 18:51:31,503::task::588::TaskManager.Task::(_updateState) Task=`0a3a9082-69c3-4671-850e-c610f30d47a8`::moving from state preparing -> state finished >Thread-248::DEBUG::2012-12-04 18:51:31,503::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-248::DEBUG::2012-12-04 18:51:31,503::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-248::DEBUG::2012-12-04 18:51:31,503::task::978::TaskManager.Task::(_decref) Task=`0a3a9082-69c3-4671-850e-c610f30d47a8`::ref 0 aborting False >Thread-249::DEBUG::2012-12-04 18:51:31,536::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-249::DEBUG::2012-12-04 18:51:31,536::task::588::TaskManager.Task::(_updateState) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::moving from state init -> state preparing >Thread-249::INFO::2012-12-04 18:51:31,537::logUtils::37::dispatcher::(wrapper) Run and protect: spmStart(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', prevID=-1, prevLVER='0', recoveryMode=None, scsiFencing='false', maxHostID=250, domVersion='3', options=None) >Thread-249::DEBUG::2012-12-04 18:51:31,537::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`72a679b4-4727-4182-81ae-b276655cade0`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-249::DEBUG::2012-12-04 18:51:31,537::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-249::DEBUG::2012-12-04 18:51:31,537::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-249::DEBUG::2012-12-04 18:51:31,537::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`72a679b4-4727-4182-81ae-b276655cade0`::Granted request >Thread-249::DEBUG::2012-12-04 18:51:31,538::task::817::TaskManager.Task::(resourceAcquired) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (exclusive) >Thread-249::DEBUG::2012-12-04 18:51:31,538::task::978::TaskManager.Task::(_decref) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::ref 1 aborting False >Thread-249::INFO::2012-12-04 18:51:31,538::logUtils::39::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >Thread-249::DEBUG::2012-12-04 18:51:31,538::task::1167::TaskManager.Task::(prepare) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::Prepare: 1 jobs exist, move to acquiring >Thread-249::DEBUG::2012-12-04 18:51:31,538::task::588::TaskManager.Task::(_updateState) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::moving from state preparing -> state acquiring >Thread-249::DEBUG::2012-12-04 18:51:31,538::task::588::TaskManager.Task::(_updateState) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::moving from state acquiring -> state queued >Thread-249::DEBUG::2012-12-04 18:51:31,539::taskManager::48::TaskManager::(_queueTask) queuing task: 8d44857f-7dd9-4ffc-bd98-89a068a8ca11 >Thread-249::DEBUG::2012-12-04 18:51:31,539::taskManager::54::TaskManager::(_queueTask) task queued: 8d44857f-7dd9-4ffc-bd98-89a068a8ca11 >Thread-9::DEBUG::2012-12-04 18:51:31,539::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-249::DEBUG::2012-12-04 18:51:31,539::task::1169::TaskManager.Task::(prepare) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::returning >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,539::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: 8d44857f-7dd9-4ffc-bd98-89a068a8ca11 running: <bound method Task.commit of <storage.task.Task instance at 0x7f7f140ec440>> with: None >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,540::task::1181::TaskManager.Task::(commit) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::committing task: 8d44857f-7dd9-4ffc-bd98-89a068a8ca11 >Thread-249::DEBUG::2012-12-04 18:51:31,540::task::978::TaskManager.Task::(_decref) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::ref 0 aborting False >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,540::task::588::TaskManager.Task::(_updateState) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::moving from state queued -> state running >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,541::task::892::TaskManager.Task::(_runJobs) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f7f140f01d0>> (args: (-1, '0', 'false', 250, 3) kwargs: {}) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,541::task::317::TaskManager.Task::(run) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f7f140f01d0>> (args: (-1, '0', 'false', 250, 3) kwargs: {}) callback None >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,541::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,541::misc::1055::SamplingMethod::(__call__) Got in to sampling method >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,547::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=562119f06d0985abd7d407725393c194ad740af6'] >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,548::misc::1063::SamplingMethod::(__call__) Returning last result >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,550::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=562119f06d0985abd7d407725393c194ad740af6'] >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::INFO::2012-12-04 18:51:31,551::safelease::188::SANLock::(acquire) Acquiring cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 1) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,589::safelease::213::SANLock::(acquire) Cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully acquired (id: 1) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,589::sp::260::Storage.StoragePool::(startSpm) spm lock acquired successfully >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,595::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=562119f06d0985abd7d407725393c194ad740af6'] >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,595::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,595::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,595::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=8dc966ae-7319-4db8-8fab-33f01a0512b6:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=3804e011c00a9fcc8c243d5afcabeb2d3d7d8e1d'] >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,602::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,602::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`f1cd8d9b-2083-42f9-a4b5-8c3f6c79a794`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,602::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,602::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,602::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`f1cd8d9b-2083-42f9-a4b5-8c3f6c79a794`::Granted request >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::INFO::2012-12-04 18:51:31,603::sp::407::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,603::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`d606621c-31ee-4329-9f6c-7c9823b9561a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,603::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'exclusive' >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,603::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'exclusive' (1 active user) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,603::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`d606621c-31ee-4329-9f6c-7c9823b9561a`::Granted request >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,604::sd::385::Storage.StorageDomain::(upgrade) Trying to upgrade domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` from version 3 to version 3 >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,604::sd::391::Storage.StorageDomain::(upgrade) No need to upgrade domain `8dc966ae-7319-4db8-8fab-33f01a0512b6`, leaving unchanged >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,604::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,604::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,604::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,604::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,605::sp::411::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,605::sp::418::Storage.StoragePool::(_upgradePool) Registering with state change event >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,605::sp::420::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,605::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,605::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,605::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,605::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,609::sp::309::Storage.StoragePool::(startSpm) ended. >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,609::task::588::TaskManager.Task::(_updateState) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::moving from state running -> state finished >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,609::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,609::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,609::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,610::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,610::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,610::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,610::task::900::TaskManager.Task::(_runJobs) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::Task.run: exit - success: result >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,610::task::978::TaskManager.Task::(_decref) Task=`8d44857f-7dd9-4ffc-bd98-89a068a8ca11`::ref 0 aborting False >8d44857f-7dd9-4ffc-bd98-89a068a8ca11::DEBUG::2012-12-04 18:51:31,610::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-251::DEBUG::2012-12-04 18:51:32,568::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-251::DEBUG::2012-12-04 18:51:32,568::task::588::TaskManager.Task::(_updateState) Task=`2956fc34-9cb4-4e14-9668-87b197f3110f`::moving from state init -> state preparing >Thread-251::INFO::2012-12-04 18:51:32,568::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='8d44857f-7dd9-4ffc-bd98-89a068a8ca11', spUUID=None, options=None) >Thread-251::DEBUG::2012-12-04 18:51:32,568::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 8d44857f-7dd9-4ffc-bd98-89a068a8ca11 >Thread-251::DEBUG::2012-12-04 18:51:32,569::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '8d44857f-7dd9-4ffc-bd98-89a068a8ca11'} >Thread-251::INFO::2012-12-04 18:51:32,569::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '8d44857f-7dd9-4ffc-bd98-89a068a8ca11'}} >Thread-251::DEBUG::2012-12-04 18:51:32,569::task::1172::TaskManager.Task::(prepare) Task=`2956fc34-9cb4-4e14-9668-87b197f3110f`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '8d44857f-7dd9-4ffc-bd98-89a068a8ca11'}} >Thread-251::DEBUG::2012-12-04 18:51:32,569::task::588::TaskManager.Task::(_updateState) Task=`2956fc34-9cb4-4e14-9668-87b197f3110f`::moving from state preparing -> state finished >Thread-251::DEBUG::2012-12-04 18:51:32,569::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-251::DEBUG::2012-12-04 18:51:32,569::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-251::DEBUG::2012-12-04 18:51:32,569::task::978::TaskManager.Task::(_decref) Task=`2956fc34-9cb4-4e14-9668-87b197f3110f`::ref 0 aborting False >Thread-252::DEBUG::2012-12-04 18:51:32,595::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-252::DEBUG::2012-12-04 18:51:32,595::task::588::TaskManager.Task::(_updateState) Task=`b61618e7-81b0-4574-b800-1ebab5366454`::moving from state init -> state preparing >Thread-252::INFO::2012-12-04 18:51:32,596::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-252::INFO::2012-12-04 18:51:32,596::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 1}} >Thread-252::DEBUG::2012-12-04 18:51:32,596::task::1172::TaskManager.Task::(prepare) Task=`b61618e7-81b0-4574-b800-1ebab5366454`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 1}} >Thread-252::DEBUG::2012-12-04 18:51:32,596::task::588::TaskManager.Task::(_updateState) Task=`b61618e7-81b0-4574-b800-1ebab5366454`::moving from state preparing -> state finished >Thread-252::DEBUG::2012-12-04 18:51:32,596::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-252::DEBUG::2012-12-04 18:51:32,596::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-252::DEBUG::2012-12-04 18:51:32,596::task::978::TaskManager.Task::(_decref) Task=`b61618e7-81b0-4574-b800-1ebab5366454`::ref 0 aborting False >Thread-253::DEBUG::2012-12-04 18:51:32,625::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-253::DEBUG::2012-12-04 18:51:32,625::task::588::TaskManager.Task::(_updateState) Task=`7d3a0c5f-42d8-422d-bafc-3dc8f255dec9`::moving from state init -> state preparing >Thread-253::INFO::2012-12-04 18:51:32,625::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='8d44857f-7dd9-4ffc-bd98-89a068a8ca11', spUUID=None, options=None) >Thread-253::DEBUG::2012-12-04 18:51:32,625::taskManager::161::TaskManager::(clearTask) Entry. taskID: 8d44857f-7dd9-4ffc-bd98-89a068a8ca11 >Thread-253::DEBUG::2012-12-04 18:51:32,625::taskManager::166::TaskManager::(clearTask) Return. >Thread-253::INFO::2012-12-04 18:51:32,626::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-253::DEBUG::2012-12-04 18:51:32,626::task::1172::TaskManager.Task::(prepare) Task=`7d3a0c5f-42d8-422d-bafc-3dc8f255dec9`::finished: None >Thread-253::DEBUG::2012-12-04 18:51:32,626::task::588::TaskManager.Task::(_updateState) Task=`7d3a0c5f-42d8-422d-bafc-3dc8f255dec9`::moving from state preparing -> state finished >Thread-253::DEBUG::2012-12-04 18:51:32,626::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-253::DEBUG::2012-12-04 18:51:32,626::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-253::DEBUG::2012-12-04 18:51:32,626::task::978::TaskManager.Task::(_decref) Task=`7d3a0c5f-42d8-422d-bafc-3dc8f255dec9`::ref 0 aborting False >Thread-254::DEBUG::2012-12-04 18:51:32,687::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-254::DEBUG::2012-12-04 18:51:32,687::task::588::TaskManager.Task::(_updateState) Task=`19b27486-a1dd-4102-948e-c6814ad2f6e0`::moving from state init -> state preparing >Thread-254::INFO::2012-12-04 18:51:32,687::logUtils::37::dispatcher::(wrapper) Run and protect: destroyStoragePool(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', hostID=1, scsiKey='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', options=None) >Thread-254::INFO::2012-12-04 18:51:32,687::hsm::925::Storage.HSM::(destroyStoragePool) spUUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb >Thread-254::DEBUG::2012-12-04 18:51:32,688::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`24c23049-17cc-4cc3-9922-2f238021a7fd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-254::DEBUG::2012-12-04 18:51:32,688::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-254::DEBUG::2012-12-04 18:51:32,688::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-254::DEBUG::2012-12-04 18:51:32,688::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`24c23049-17cc-4cc3-9922-2f238021a7fd`::Granted request >Thread-254::DEBUG::2012-12-04 18:51:32,688::task::817::TaskManager.Task::(resourceAcquired) Task=`19b27486-a1dd-4102-948e-c6814ad2f6e0`::_resourcesAcquired: Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb (exclusive) >Thread-254::DEBUG::2012-12-04 18:51:32,688::task::978::TaskManager.Task::(_decref) Task=`19b27486-a1dd-4102-948e-c6814ad2f6e0`::ref 1 aborting False >Thread-254::DEBUG::2012-12-04 18:51:32,689::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`0455af9e-6675-455c-8816-74025eee60ff`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-254::DEBUG::2012-12-04 18:51:32,689::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'exclusive' >Thread-254::DEBUG::2012-12-04 18:51:32,689::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'exclusive' (1 active user) >Thread-254::DEBUG::2012-12-04 18:51:32,689::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`0455af9e-6675-455c-8816-74025eee60ff`::Granted request >Thread-254::DEBUG::2012-12-04 18:51:32,690::task::817::TaskManager.Task::(resourceAcquired) Task=`19b27486-a1dd-4102-948e-c6814ad2f6e0`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (exclusive) >Thread-254::DEBUG::2012-12-04 18:51:32,690::task::978::TaskManager.Task::(_decref) Task=`19b27486-a1dd-4102-948e-c6814ad2f6e0`::ref 1 aborting False >Thread-254::WARNING::2012-12-04 18:51:32,690::sp::933::Storage.StoragePool::(forcedDetachSD) Force detaching domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` >Thread-254::DEBUG::2012-12-04 18:51:32,690::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-254::DEBUG::2012-12-04 18:51:32,690::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-254::DEBUG::2012-12-04 18:51:32,691::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=cc20007a5b7979672f5eee004789cac28891ad33'] >Thread-254::DEBUG::2012-12-04 18:51:32,694::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-254::DEBUG::2012-12-04 18:51:32,695::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-254::DEBUG::2012-12-04 18:51:32,695::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-254::DEBUG::2012-12-04 18:51:32,695::sp::1487::Storage.StoragePool::(updateMonitoringThreads) sp `ac5d6452-d8fe-4d69-8d98-d99cd5a494bb` stopped monitoring domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` >Thread-254::DEBUG::2012-12-04 18:51:32,695::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-254::DEBUG::2012-12-04 18:51:32,695::sp::941::Storage.StoragePool::(forcedDetachSD) Force detach for domain `8dc966ae-7319-4db8-8fab-33f01a0512b6` is done >Thread-254::DEBUG::2012-12-04 18:51:32,699::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=1', 'POOL_UUID=ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Master', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=cc20007a5b7979672f5eee004789cac28891ad33'] >Thread-254::DEBUG::2012-12-04 18:51:32,699::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-254::DEBUG::2012-12-04 18:51:32,699::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-254::DEBUG::2012-12-04 18:51:32,700::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-254::DEBUG::2012-12-04 18:51:32,700::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-254::DEBUG::2012-12-04 18:51:32,700::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=1', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=77bd860b33651c16199d68d3bb9165ea8b52bc44'] >Thread-254::DEBUG::2012-12-04 18:51:32,704::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-254::DEBUG::2012-12-04 18:51:32,704::sp::319::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process >Thread-254::DEBUG::2012-12-04 18:51:32,705::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`5c134955-6e10-44d8-9d25-6d268e682ba7`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-254::DEBUG::2012-12-04 18:51:32,705::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' for lock type 'exclusive' >Thread-254::DEBUG::2012-12-04 18:51:32,705::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free. Now locking as 'exclusive' (1 active user) >Thread-254::DEBUG::2012-12-04 18:51:32,705::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb`ReqID=`5c134955-6e10-44d8-9d25-6d268e682ba7`::Granted request >Thread-254::DEBUG::2012-12-04 18:51:32,705::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-254::DEBUG::2012-12-04 18:51:32,706::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-254::DEBUG::2012-12-04 18:51:32,706::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-254::DEBUG::2012-12-04 18:51:32,706::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-254::DEBUG::2012-12-04 18:51:32,706::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-254::DEBUG::2012-12-04 18:51:32,706::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-254::DEBUG::2012-12-04 18:51:32,707::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=1', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=aae0928c3942a87d6e1c59f6da77937297010165'] >Thread-254::DEBUG::2012-12-04 18:51:32,710::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-254::INFO::2012-12-04 18:51:32,710::safelease::218::SANLock::(release) Releasing cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 (id: 1) >Thread-254::DEBUG::2012-12-04 18:51:32,712::safelease::229::SANLock::(release) Cluster lock for domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 successfully released (id: 1) >Thread-254::INFO::2012-12-04 18:51:32,712::sp::668::Storage.StoragePool::(disconnect) Disconnect from the storage pool ac5d6452-d8fe-4d69-8d98-d99cd5a494bb >Thread-254::INFO::2012-12-04 18:51:32,712::logUtils::39::dispatcher::(wrapper) Run and protect: destroyStoragePool, Return response: True >Thread-254::DEBUG::2012-12-04 18:51:32,713::task::1172::TaskManager.Task::(prepare) Task=`19b27486-a1dd-4102-948e-c6814ad2f6e0`::finished: True >Thread-254::DEBUG::2012-12-04 18:51:32,713::task::588::TaskManager.Task::(_updateState) Task=`19b27486-a1dd-4102-948e-c6814ad2f6e0`::moving from state preparing -> state finished >Thread-254::DEBUG::2012-12-04 18:51:32,713::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>, 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb': < ResourceRef 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', isValid: 'True' obj: 'None'>} >Thread-254::DEBUG::2012-12-04 18:51:32,713::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-254::DEBUG::2012-12-04 18:51:32,713::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-254::DEBUG::2012-12-04 18:51:32,713::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-254::DEBUG::2012-12-04 18:51:32,714::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-254::DEBUG::2012-12-04 18:51:32,714::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-254::DEBUG::2012-12-04 18:51:32,714::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' >Thread-254::DEBUG::2012-12-04 18:51:32,714::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' (0 active users) >Thread-254::DEBUG::2012-12-04 18:51:32,714::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb' is free, finding out if anyone is waiting for it. >Thread-254::DEBUG::2012-12-04 18:51:32,714::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', Clearing records. >Thread-254::DEBUG::2012-12-04 18:51:32,715::task::978::TaskManager.Task::(_decref) Task=`19b27486-a1dd-4102-948e-c6814ad2f6e0`::ref 0 aborting False >Thread-255::DEBUG::2012-12-04 18:51:32,784::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-255::DEBUG::2012-12-04 18:51:32,784::task::588::TaskManager.Task::(_updateState) Task=`03209cd9-694a-4107-934c-7d6253ec7d52`::moving from state init -> state preparing >Thread-255::INFO::2012-12-04 18:51:32,784::logUtils::37::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-255::DEBUG::2012-12-04 18:51:32,785::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604' (cwd None) >Thread-255::DEBUG::2012-12-04 18:51:32,797::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-255::DEBUG::2012-12-04 18:51:32,797::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-255::DEBUG::2012-12-04 18:51:32,798::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-255::DEBUG::2012-12-04 18:51:32,798::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-255::DEBUG::2012-12-04 18:51:32,798::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-255::DEBUG::2012-12-04 18:51:32,807::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-255::DEBUG::2012-12-04 18:51:32,807::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-255::DEBUG::2012-12-04 18:51:34,641::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-255::DEBUG::2012-12-04 18:51:34,671::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-255::DEBUG::2012-12-04 18:51:34,671::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-255::DEBUG::2012-12-04 18:51:34,671::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-255::DEBUG::2012-12-04 18:51:34,671::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-255::DEBUG::2012-12-04 18:51:34,672::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-255::DEBUG::2012-12-04 18:51:34,672::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-255::DEBUG::2012-12-04 18:51:34,672::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-255::DEBUG::2012-12-04 18:51:34,672::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-255::INFO::2012-12-04 18:51:34,672::logUtils::39::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-255::DEBUG::2012-12-04 18:51:34,673::task::1172::TaskManager.Task::(prepare) Task=`03209cd9-694a-4107-934c-7d6253ec7d52`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-255::DEBUG::2012-12-04 18:51:34,673::task::588::TaskManager.Task::(_updateState) Task=`03209cd9-694a-4107-934c-7d6253ec7d52`::moving from state preparing -> state finished >Thread-255::DEBUG::2012-12-04 18:51:34,673::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-255::DEBUG::2012-12-04 18:51:34,673::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-255::DEBUG::2012-12-04 18:51:34,673::task::978::TaskManager.Task::(_decref) Task=`03209cd9-694a-4107-934c-7d6253ec7d52`::ref 0 aborting False >Thread-257::DEBUG::2012-12-04 18:51:34,727::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-257::DEBUG::2012-12-04 18:51:34,728::task::588::TaskManager.Task::(_updateState) Task=`c0a8c1c2-781c-46c7-9801-f62b2554a068`::moving from state init -> state preparing >Thread-257::INFO::2012-12-04 18:51:34,728::logUtils::37::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', hostID=1, scsiKey='ac5d6452-d8fe-4d69-8d98-d99cd5a494bb', remove=False, options=None) >Thread-257::WARNING::2012-12-04 18:51:34,728::hsm::892::Storage.HSM::(disconnectStoragePool) disconnect sp: ac5d6452-d8fe-4d69-8d98-d99cd5a494bb failed. Known pools {} >Thread-257::INFO::2012-12-04 18:51:34,728::logUtils::39::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: None >Thread-257::DEBUG::2012-12-04 18:51:34,728::task::1172::TaskManager.Task::(prepare) Task=`c0a8c1c2-781c-46c7-9801-f62b2554a068`::finished: None >Thread-257::DEBUG::2012-12-04 18:51:34,728::task::588::TaskManager.Task::(_updateState) Task=`c0a8c1c2-781c-46c7-9801-f62b2554a068`::moving from state preparing -> state finished >Thread-257::DEBUG::2012-12-04 18:51:34,729::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-257::DEBUG::2012-12-04 18:51:34,729::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-257::DEBUG::2012-12-04 18:51:34,729::task::978::TaskManager.Task::(_decref) Task=`c0a8c1c2-781c-46c7-9801-f62b2554a068`::ref 0 aborting False >Thread-258::DEBUG::2012-12-04 18:51:35,651::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-258::DEBUG::2012-12-04 18:51:35,651::task::588::TaskManager.Task::(_updateState) Task=`8646d9d4-7d78-42b4-9c45-9252d959918c`::moving from state init -> state preparing >Thread-258::INFO::2012-12-04 18:51:35,651::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-258::INFO::2012-12-04 18:51:35,651::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-258::DEBUG::2012-12-04 18:51:35,652::task::1172::TaskManager.Task::(prepare) Task=`8646d9d4-7d78-42b4-9c45-9252d959918c`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-258::DEBUG::2012-12-04 18:51:35,652::task::588::TaskManager.Task::(_updateState) Task=`8646d9d4-7d78-42b4-9c45-9252d959918c`::moving from state preparing -> state finished >Thread-258::DEBUG::2012-12-04 18:51:35,652::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-258::DEBUG::2012-12-04 18:51:35,652::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-258::DEBUG::2012-12-04 18:51:35,652::task::978::TaskManager.Task::(_decref) Task=`8646d9d4-7d78-42b4-9c45-9252d959918c`::ref 0 aborting False >Thread-259::DEBUG::2012-12-04 18:51:35,673::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-259::DEBUG::2012-12-04 18:51:35,673::task::588::TaskManager.Task::(_updateState) Task=`70483ef3-9490-429d-b5bf-e0606f4b4298`::moving from state init -> state preparing >Thread-259::INFO::2012-12-04 18:51:35,673::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-259::DEBUG::2012-12-04 18:51:35,677::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604 /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604' (cwd None) >Thread-259::DEBUG::2012-12-04 18:51:35,733::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-259::DEBUG::2012-12-04 18:51:35,733::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-259::DEBUG::2012-12-04 18:51:35,734::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-259::DEBUG::2012-12-04 18:51:35,734::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-259::DEBUG::2012-12-04 18:51:35,734::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-259::DEBUG::2012-12-04 18:51:35,734::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-259::INFO::2012-12-04 18:51:35,734::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-259::DEBUG::2012-12-04 18:51:35,735::task::1172::TaskManager.Task::(prepare) Task=`70483ef3-9490-429d-b5bf-e0606f4b4298`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-259::DEBUG::2012-12-04 18:51:35,735::task::588::TaskManager.Task::(_updateState) Task=`70483ef3-9490-429d-b5bf-e0606f4b4298`::moving from state preparing -> state finished >Thread-259::DEBUG::2012-12-04 18:51:35,735::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-259::DEBUG::2012-12-04 18:51:35,735::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-259::DEBUG::2012-12-04 18:51:35,735::task::978::TaskManager.Task::(_decref) Task=`70483ef3-9490-429d-b5bf-e0606f4b4298`::ref 0 aborting False >Thread-260::DEBUG::2012-12-04 18:51:35,767::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-260::DEBUG::2012-12-04 18:51:35,768::task::588::TaskManager.Task::(_updateState) Task=`18c4f825-5a04-4a7c-b9af-6f8b84c795bd`::moving from state init -> state preparing >Thread-260::INFO::2012-12-04 18:51:35,768::logUtils::37::dispatcher::(wrapper) Run and protect: formatStorageDomain(sdUUID='8dc966ae-7319-4db8-8fab-33f01a0512b6', autoDetach=False, options=None) >Thread-260::DEBUG::2012-12-04 18:51:35,768::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`a8c0cbc1-ee7e-4d21-a0f0-6fd5420519ba`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-260::DEBUG::2012-12-04 18:51:35,768::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' for lock type 'exclusive' >Thread-260::DEBUG::2012-12-04 18:51:35,769::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free. Now locking as 'exclusive' (1 active user) >Thread-260::DEBUG::2012-12-04 18:51:35,769::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6`ReqID=`a8c0cbc1-ee7e-4d21-a0f0-6fd5420519ba`::Granted request >Thread-260::DEBUG::2012-12-04 18:51:35,769::task::817::TaskManager.Task::(resourceAcquired) Task=`18c4f825-5a04-4a7c-b9af-6f8b84c795bd`::_resourcesAcquired: Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6 (exclusive) >Thread-260::DEBUG::2012-12-04 18:51:35,769::task::978::TaskManager.Task::(_decref) Task=`18c4f825-5a04-4a7c-b9af-6f8b84c795bd`::ref 1 aborting False >Thread-260::DEBUG::2012-12-04 18:51:35,769::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-260::DEBUG::2012-12-04 18:51:35,770::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-260::DEBUG::2012-12-04 18:51:35,770::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-260::DEBUG::2012-12-04 18:51:35,770::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-260::DEBUG::2012-12-04 18:51:35,770::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-260::DEBUG::2012-12-04 18:51:35,779::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-260::DEBUG::2012-12-04 18:51:35,779::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-260::DEBUG::2012-12-04 18:51:37,613::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-260::DEBUG::2012-12-04 18:51:37,645::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-260::DEBUG::2012-12-04 18:51:37,645::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-260::DEBUG::2012-12-04 18:51:37,645::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-260::DEBUG::2012-12-04 18:51:37,645::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-260::DEBUG::2012-12-04 18:51:37,646::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-260::DEBUG::2012-12-04 18:51:37,646::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-260::DEBUG::2012-12-04 18:51:37,646::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-260::DEBUG::2012-12-04 18:51:37,646::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-260::DEBUG::2012-12-04 18:51:37,646::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-260::DEBUG::2012-12-04 18:51:37,647::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 8dc966ae-7319-4db8-8fab-33f01a0512b6' (cwd None) >Thread-260::DEBUG::2012-12-04 18:51:37,663::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found\n'; <rc> = 5 >Thread-260::WARNING::2012-12-04 18:51:37,664::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "8dc966ae-7319-4db8-8fab-33f01a0512b6" not found'] >Thread-260::DEBUG::2012-12-04 18:51:37,664::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-260::DEBUG::2012-12-04 18:51:37,675::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604/8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-260::DEBUG::2012-12-04 18:51:37,675::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-260::DEBUG::2012-12-04 18:51:37,680::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=1', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=aae0928c3942a87d6e1c59f6da77937297010165'] >Thread-260::DEBUG::2012-12-04 18:51:37,682::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-260::WARNING::2012-12-04 18:51:37,682::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_imageNS already registered >Thread-260::WARNING::2012-12-04 18:51:37,682::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8dc966ae-7319-4db8-8fab-33f01a0512b6_volumeNS already registered >Thread-260::DEBUG::2012-12-04 18:51:37,685::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=DataDomainRestUpd', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=DefaultRest', 'POOL_DOMAINS=', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=1', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'ROLE=Regular', 'SDUUID=8dc966ae-7319-4db8-8fab-33f01a0512b6', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=aae0928c3942a87d6e1c59f6da77937297010165'] >Thread-260::INFO::2012-12-04 18:51:37,685::fileSD::331::Storage.StorageDomain::(format) Formatting domain 8dc966ae-7319-4db8-8fab-33f01a0512b6 >Thread-260::INFO::2012-12-04 18:51:37,816::logUtils::39::dispatcher::(wrapper) Run and protect: formatStorageDomain, Return response: None >Thread-260::DEBUG::2012-12-04 18:51:37,816::task::1172::TaskManager.Task::(prepare) Task=`18c4f825-5a04-4a7c-b9af-6f8b84c795bd`::finished: None >Thread-260::DEBUG::2012-12-04 18:51:37,816::task::588::TaskManager.Task::(_updateState) Task=`18c4f825-5a04-4a7c-b9af-6f8b84c795bd`::moving from state preparing -> state finished >Thread-260::DEBUG::2012-12-04 18:51:37,817::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6': < ResourceRef 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', isValid: 'True' obj: 'None'>} >Thread-260::DEBUG::2012-12-04 18:51:37,817::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-260::DEBUG::2012-12-04 18:51:37,817::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' >Thread-260::DEBUG::2012-12-04 18:51:37,817::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' (0 active users) >Thread-260::DEBUG::2012-12-04 18:51:37,817::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6' is free, finding out if anyone is waiting for it. >Thread-260::DEBUG::2012-12-04 18:51:37,817::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8dc966ae-7319-4db8-8fab-33f01a0512b6', Clearing records. >Thread-260::DEBUG::2012-12-04 18:51:37,818::task::978::TaskManager.Task::(_decref) Task=`18c4f825-5a04-4a7c-b9af-6f8b84c795bd`::ref 0 aborting False >Thread-264::DEBUG::2012-12-04 18:51:37,859::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-264::DEBUG::2012-12-04 18:51:37,860::task::588::TaskManager.Task::(_updateState) Task=`81e441e4-f496-4643-ab7b-d84064420235`::moving from state init -> state preparing >Thread-264::INFO::2012-12-04 18:51:37,860::logUtils::37::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_2012120418415856604', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'bcf0371b-5b66-4293-8000-445d692526a5', 'port': ''}], options=None) >Thread-264::DEBUG::2012-12-04 18:51:37,860::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__2012120418415856604' (cwd None) >Thread-264::DEBUG::2012-12-04 18:51:37,888::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-264::DEBUG::2012-12-04 18:51:37,888::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-264::DEBUG::2012-12-04 18:51:37,888::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-264::DEBUG::2012-12-04 18:51:37,889::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-264::DEBUG::2012-12-04 18:51:37,889::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-264::DEBUG::2012-12-04 18:51:37,898::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-264::DEBUG::2012-12-04 18:51:37,898::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-265::DEBUG::2012-12-04 18:51:38,293::task::588::TaskManager.Task::(_updateState) Task=`49a2f00a-e191-4e7f-acb5-03423ce2d4ce`::moving from state init -> state preparing >Thread-265::INFO::2012-12-04 18:51:38,293::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-265::INFO::2012-12-04 18:51:38,293::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-265::DEBUG::2012-12-04 18:51:38,293::task::1172::TaskManager.Task::(prepare) Task=`49a2f00a-e191-4e7f-acb5-03423ce2d4ce`::finished: {} >Thread-265::DEBUG::2012-12-04 18:51:38,293::task::588::TaskManager.Task::(_updateState) Task=`49a2f00a-e191-4e7f-acb5-03423ce2d4ce`::moving from state preparing -> state finished >Thread-265::DEBUG::2012-12-04 18:51:38,293::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-265::DEBUG::2012-12-04 18:51:38,294::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-265::DEBUG::2012-12-04 18:51:38,294::task::978::TaskManager.Task::(_decref) Task=`49a2f00a-e191-4e7f-acb5-03423ce2d4ce`::ref 0 aborting False >Thread-264::DEBUG::2012-12-04 18:51:39,732::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-264::DEBUG::2012-12-04 18:51:39,761::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-264::DEBUG::2012-12-04 18:51:39,761::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-264::DEBUG::2012-12-04 18:51:39,761::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-264::DEBUG::2012-12-04 18:51:39,762::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-264::DEBUG::2012-12-04 18:51:39,762::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-264::DEBUG::2012-12-04 18:51:39,762::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-264::DEBUG::2012-12-04 18:51:39,762::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-264::DEBUG::2012-12-04 18:51:39,762::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-264::INFO::2012-12-04 18:51:39,762::logUtils::39::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-264::DEBUG::2012-12-04 18:51:39,763::task::1172::TaskManager.Task::(prepare) Task=`81e441e4-f496-4643-ab7b-d84064420235`::finished: {'statuslist': [{'status': 0, 'id': 'bcf0371b-5b66-4293-8000-445d692526a5'}]} >Thread-264::DEBUG::2012-12-04 18:51:39,763::task::588::TaskManager.Task::(_updateState) Task=`81e441e4-f496-4643-ab7b-d84064420235`::moving from state preparing -> state finished >Thread-264::DEBUG::2012-12-04 18:51:39,763::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-264::DEBUG::2012-12-04 18:51:39,763::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-264::DEBUG::2012-12-04 18:51:39,763::task::978::TaskManager.Task::(_decref) Task=`81e441e4-f496-4643-ab7b-d84064420235`::ref 0 aborting False >Thread-268::DEBUG::2012-12-04 18:51:42,532::task::588::TaskManager.Task::(_updateState) Task=`a86d9d41-bb74-49fd-a137-c14660ec477b`::moving from state init -> state preparing >Thread-268::INFO::2012-12-04 18:51:42,532::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-268::INFO::2012-12-04 18:51:42,532::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-268::DEBUG::2012-12-04 18:51:42,532::task::1172::TaskManager.Task::(prepare) Task=`a86d9d41-bb74-49fd-a137-c14660ec477b`::finished: {} >Thread-268::DEBUG::2012-12-04 18:51:42,532::task::588::TaskManager.Task::(_updateState) Task=`a86d9d41-bb74-49fd-a137-c14660ec477b`::moving from state preparing -> state finished >Thread-268::DEBUG::2012-12-04 18:51:42,532::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-268::DEBUG::2012-12-04 18:51:42,533::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-268::DEBUG::2012-12-04 18:51:42,533::task::978::TaskManager.Task::(_decref) Task=`a86d9d41-bb74-49fd-a137-c14660ec477b`::ref 0 aborting False >MainThread::INFO::2012-12-05 02:31:21,087::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-05 02:31:21,088::task::588::TaskManager.Task::(_updateState) Task=`136d310c-a109-43ca-80c5-fa8d6ad4516f`::moving from state init -> state preparing >MainThread::INFO::2012-12-05 02:31:21,088::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-05 02:31:21,088::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-05 02:31:21,123::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-05 02:31:21,123::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-05 02:31:21,123::task::1172::TaskManager.Task::(prepare) Task=`136d310c-a109-43ca-80c5-fa8d6ad4516f`::finished: None >MainThread::DEBUG::2012-12-05 02:31:21,123::task::588::TaskManager.Task::(_updateState) Task=`136d310c-a109-43ca-80c5-fa8d6ad4516f`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-05 02:31:21,123::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-05 02:31:21,123::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-05 02:31:21,123::task::978::TaskManager.Task::(_decref) Task=`136d310c-a109-43ca-80c5-fa8d6ad4516f`::ref 0 aborting False >MainThread::INFO::2012-12-05 02:31:21,124::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-05 02:31:21,124::vdsm::78::vds::(run) <_MainThread(MainThread, started 140185368868672)> >MainThread::INFO::2012-12-05 02:31:21,124::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140185138431744)> >MainThread::INFO::2012-12-05 02:31:21,124::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140185006487296)> >MainThread::INFO::2012-12-05 02:31:21,124::vdsm::78::vds::(run) <WorkerThread(8d44857f-7dd9-4ffc-bd98-89a068a8ca11, started daemon 140184502920960)> >MainThread::INFO::2012-12-05 02:31:21,124::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140185014880000)> >MainThread::INFO::2012-12-05 02:31:21,124::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140184494528256)> >MainThread::INFO::2012-12-05 02:31:21,124::vdsm::78::vds::(run) <WorkerThread(6fae11e5-c4fc-4bd9-ac1f-d5a62b0cfedf, started daemon 140185031665408)> >MainThread::INFO::2012-12-05 02:31:21,125::vdsm::78::vds::(run) <WorkerThread(dbbc5edc-4a3f-452b-a086-0b9e22b52784, started daemon 140184511313664)> >MainThread::INFO::2012-12-05 02:31:21,125::vdsm::78::vds::(run) <Thread(Thread-30, started daemon 140183949264640)> >MainThread::INFO::2012-12-05 02:31:21,125::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140185040058112)> >MainThread::INFO::2012-12-05 02:31:21,125::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140184998094592)> >MainThread::INFO::2012-12-05 02:31:21,125::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140184460957440)> >MainThread::INFO::2012-12-05 02:31:21,125::vdsm::78::vds::(run) <WorkerThread(b534ef17-f251-464a-a83b-b67dcc95922c, started daemon 140184989701888)> >MainThread::INFO::2012-12-05 02:31:21,125::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140185023272704)> >MainThread::INFO::2012-12-05 02:31:22,382::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-05 02:31:22,547::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-05 02:31:22,547::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-05 02:31:22,573::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 1892' (cwd None) >MainThread::DEBUG::2012-12-05 02:31:22,595::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 1892: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-05 02:31:22,595::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-05 02:31:22,595::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py b2e1c489-5fe3-4320-bbc9-54fd7ad12009 17547' (cwd None) >MainThread::DEBUG::2012-12-05 02:31:22,730::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-05 02:31:22,730::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-05 02:31:22,730::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-05 02:31:22,730::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-05 02:31:22,731::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-05 02:31:22,731::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-05 02:31:22,732::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-05 02:31:24,600::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-05 02:31:24,606::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-05 02:31:24,607::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-05 02:31:24,607::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-05 02:31:24,627::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-05 02:31:24,628::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-05 02:31:24,629::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-05 02:31:24,645::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-05 02:31:24,646::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-05 02:31:24,646::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-05 02:31:24,646::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-05 02:31:24,661::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-05 02:31:24,662::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-05 02:31:24,662::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-05 02:31:24,677::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-05 02:31:24,678::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-05 02:31:24,679::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-05 02:31:24,679::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-05 02:31:24,680::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-05 02:31:24,680::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-05 02:31:24,680::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-05 02:31:24,688::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-05 02:31:24,692::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-05 02:31:24,692::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-05 02:31:24,697::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-05 02:31:24,697::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-05 02:31:24,698::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-05 02:31:24,701::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-05 02:31:26,527::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-05 02:31:26,567::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-05 02:31:26,567::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 02:31:26,568::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 02:31:26,568::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 02:31:26,568::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 02:31:26,568::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 02:31:26,568::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 02:31:26,569::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-05 02:31:26,569::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-05 02:31:26,572::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-05 02:31:26,572::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-05 02:31:26,573::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-05 02:31:26,573::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-05 07:42:33,279::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-05 07:42:33,280::task::588::TaskManager.Task::(_updateState) Task=`35ee6a5d-c0ff-4042-8b89-3abc707927ce`::moving from state init -> state preparing >MainThread::INFO::2012-12-05 07:42:33,281::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-05 07:42:33,281::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-05 07:42:33,307::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-05 07:42:33,307::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-05 07:42:33,307::task::1172::TaskManager.Task::(prepare) Task=`35ee6a5d-c0ff-4042-8b89-3abc707927ce`::finished: None >MainThread::DEBUG::2012-12-05 07:42:33,307::task::588::TaskManager.Task::(_updateState) Task=`35ee6a5d-c0ff-4042-8b89-3abc707927ce`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-05 07:42:33,307::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-05 07:42:33,307::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-05 07:42:33,308::task::978::TaskManager.Task::(_decref) Task=`35ee6a5d-c0ff-4042-8b89-3abc707927ce`::ref 0 aborting False >MainThread::INFO::2012-12-05 07:42:33,308::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-05 07:42:33,308::vdsm::78::vds::(run) <_MainThread(MainThread, started 140044388210496)> >MainThread::INFO::2012-12-05 07:42:33,308::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140044157773568)> >MainThread::INFO::2012-12-05 07:42:33,308::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-05 07:42:33,308::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140042911610624)> >MainThread::INFO::2012-12-05 07:42:33,308::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140044018890496)> >MainThread::INFO::2012-12-05 07:42:33,308::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140043440088832)> >MainThread::INFO::2012-12-05 07:42:33,309::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140044128405248)> >MainThread::INFO::2012-12-05 07:42:33,309::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140043406518016)> >MainThread::INFO::2012-12-05 07:42:33,309::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140044044068608)> >MainThread::INFO::2012-12-05 07:42:33,309::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140043448481536)> >MainThread::INFO::2012-12-05 07:42:33,309::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140044052461312)> >MainThread::INFO::2012-12-05 07:42:33,309::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140044027283200)> >MainThread::INFO::2012-12-05 07:42:33,309::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140044010497792)> >MainThread::INFO::2012-12-05 07:42:33,309::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140044002105088)> >MainThread::INFO::2012-12-05 07:42:33,310::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140044035675904)> >MainThread::INFO::2012-12-05 07:42:34,759::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-05 07:42:34,925::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-05 07:42:34,925::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-05 07:42:34,945::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 17568' (cwd None) >MainThread::DEBUG::2012-12-05 07:42:34,955::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 17568: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-05 07:42:34,955::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-05 07:42:34,955::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py ee18abdb-0cbe-470d-a4f8-036a8bd1f114 1512' (cwd None) >MainThread::DEBUG::2012-12-05 07:42:35,091::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-05 07:42:35,091::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-05 07:42:35,091::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-05 07:42:35,091::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-05 07:42:35,092::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-05 07:42:35,092::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-05 07:42:35,093::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-05 07:42:36,961::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-05 07:42:36,967::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-05 07:42:36,967::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-05 07:42:36,968::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-05 07:42:36,989::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-05 07:42:36,990::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-05 07:42:36,991::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-05 07:42:37,008::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-05 07:42:37,008::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-05 07:42:37,009::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-05 07:42:37,009::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-05 07:42:37,025::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-05 07:42:37,025::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-05 07:42:37,026::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-05 07:42:37,041::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-05 07:42:37,042::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-05 07:42:37,043::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-05 07:42:37,043::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-05 07:42:37,044::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-05 07:42:37,044::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-05 07:42:37,044::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-05 07:42:37,053::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-05 07:42:37,057::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-05 07:42:37,057::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-05 07:42:37,066::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-05 07:42:37,067::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-05 07:42:37,068::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-05 07:42:37,070::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-05 07:42:38,892::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-05 07:42:38,935::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-05 07:42:38,936::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 07:42:38,936::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 07:42:38,936::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 07:42:38,936::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 07:42:38,936::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 07:42:38,937::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 07:42:38,937::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-05 07:42:38,937::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-05 07:42:38,941::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-05 07:42:38,941::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-05 07:42:38,941::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-05 07:42:38,941::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-05 13:40:42,863::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-05 13:40:42,864::task::588::TaskManager.Task::(_updateState) Task=`44bf724e-ccb2-4e83-a388-9718990a6c01`::moving from state init -> state preparing >MainThread::INFO::2012-12-05 13:40:42,864::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-05 13:40:42,864::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-05 13:40:42,891::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-05 13:40:42,891::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-05 13:40:42,891::task::1172::TaskManager.Task::(prepare) Task=`44bf724e-ccb2-4e83-a388-9718990a6c01`::finished: None >MainThread::DEBUG::2012-12-05 13:40:42,891::task::588::TaskManager.Task::(_updateState) Task=`44bf724e-ccb2-4e83-a388-9718990a6c01`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-05 13:40:42,891::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-05 13:40:42,891::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-05 13:40:42,892::task::978::TaskManager.Task::(_decref) Task=`44bf724e-ccb2-4e83-a388-9718990a6c01`::ref 0 aborting False >MainThread::INFO::2012-12-05 13:40:42,892::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-05 13:40:42,892::vdsm::78::vds::(run) <_MainThread(MainThread, started 140521421752128)> >MainThread::INFO::2012-12-05 13:40:42,892::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140521191315200)> >MainThread::INFO::2012-12-05 13:40:42,892::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140520894478080)> >MainThread::INFO::2012-12-05 13:40:42,892::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140520877692672)> >MainThread::INFO::2012-12-05 13:40:42,892::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140521161946880)> >MainThread::INFO::2012-12-05 13:40:42,892::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140520902870784)> >MainThread::INFO::2012-12-05 13:40:42,893::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140521145161472)> >MainThread::INFO::2012-12-05 13:40:42,893::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-05 13:40:42,893::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140520282105600)> >MainThread::INFO::2012-12-05 13:40:42,893::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140520919656192)> >MainThread::INFO::2012-12-05 13:40:42,893::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140520928048896)> >MainThread::INFO::2012-12-05 13:40:42,893::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140520886085376)> >MainThread::INFO::2012-12-05 13:40:42,893::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140520298891008)> >MainThread::INFO::2012-12-05 13:40:42,893::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140520911263488)> >MainThread::INFO::2012-12-05 13:40:42,894::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140521153554176)> >MainThread::INFO::2012-12-05 13:40:44,465::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-05 13:40:44,628::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-05 13:40:44,629::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-05 13:40:44,649::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 1533' (cwd None) >MainThread::DEBUG::2012-12-05 13:40:44,659::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 1533: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-05 13:40:44,659::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-05 13:40:44,659::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py d5ed53d0-463a-4d41-b9b0-5f82f63751bc 12439' (cwd None) >MainThread::DEBUG::2012-12-05 13:40:44,793::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-05 13:40:44,793::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-05 13:40:44,793::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-05 13:40:44,793::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-05 13:40:44,794::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-05 13:40:44,794::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-05 13:40:44,795::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-05 13:40:46,664::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-05 13:40:46,670::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-05 13:40:46,671::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-05 13:40:46,671::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-05 13:40:46,692::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-05 13:40:46,693::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-05 13:40:46,693::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-05 13:40:46,710::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-05 13:40:46,710::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-05 13:40:46,710::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-05 13:40:46,711::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-05 13:40:46,727::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-05 13:40:46,727::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-05 13:40:46,727::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-05 13:40:46,742::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-05 13:40:46,743::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-05 13:40:46,744::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-05 13:40:46,744::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-05 13:40:46,745::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-05 13:40:46,745::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-05 13:40:46,745::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-05 13:40:46,754::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-05 13:40:46,757::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-05 13:40:46,757::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-05 13:40:46,767::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-05 13:40:46,768::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-05 13:40:46,768::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-05 13:40:46,771::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-05 13:40:48,593::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-05 13:40:48,638::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-05 13:40:48,639::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 13:40:48,639::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 13:40:48,639::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 13:40:48,639::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 13:40:48,639::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 13:40:48,640::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 13:40:48,640::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-05 13:40:48,640::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-05 13:40:48,644::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-05 13:40:48,644::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-05 13:40:48,644::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-05 13:40:48,645::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-05 19:35:55,139::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-05 19:35:55,140::task::588::TaskManager.Task::(_updateState) Task=`8265487c-30d6-4d19-9aec-51ab127e2d96`::moving from state init -> state preparing >MainThread::INFO::2012-12-05 19:35:55,140::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-05 19:35:55,140::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-05 19:35:55,166::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-05 19:35:55,166::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-05 19:35:55,166::task::1172::TaskManager.Task::(prepare) Task=`8265487c-30d6-4d19-9aec-51ab127e2d96`::finished: None >MainThread::DEBUG::2012-12-05 19:35:55,167::task::588::TaskManager.Task::(_updateState) Task=`8265487c-30d6-4d19-9aec-51ab127e2d96`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-05 19:35:55,167::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-05 19:35:55,167::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-05 19:35:55,167::task::978::TaskManager.Task::(_decref) Task=`8265487c-30d6-4d19-9aec-51ab127e2d96`::ref 0 aborting False >MainThread::INFO::2012-12-05 19:35:55,167::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 12 other threads... >MainThread::INFO::2012-12-05 19:35:55,167::vdsm::78::vds::(run) <_MainThread(MainThread, started 140365790492480)> >MainThread::INFO::2012-12-05 19:35:55,167::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140365560055552)> >MainThread::INFO::2012-12-05 19:35:55,168::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140365269022464)> >MainThread::INFO::2012-12-05 19:35:55,168::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140365252237056)> >MainThread::INFO::2012-12-05 19:35:55,168::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140365513901824)> >MainThread::INFO::2012-12-05 19:35:55,168::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140365294200576)> >MainThread::INFO::2012-12-05 19:35:55,168::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140365285807872)> >MainThread::INFO::2012-12-05 19:35:55,168::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140365530687232)> >MainThread::INFO::2012-12-05 19:35:55,168::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140365302593280)> >MainThread::INFO::2012-12-05 19:35:55,168::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140364673435392)> >MainThread::INFO::2012-12-05 19:35:55,169::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140365260629760)> >MainThread::INFO::2012-12-05 19:35:55,169::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140365277415168)> >MainThread::INFO::2012-12-05 19:35:55,169::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140365522294528)> >MainThread::INFO::2012-12-05 19:35:56,028::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-05 19:35:56,192::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-05 19:35:56,192::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-05 19:35:56,220::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 12460' (cwd None) >MainThread::DEBUG::2012-12-05 19:35:56,229::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 12460: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-05 19:35:56,230::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-05 19:35:56,230::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 0f4c19ab-7078-4de6-8bb4-9c54f848b4ec 22735' (cwd None) >MainThread::DEBUG::2012-12-05 19:35:56,365::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-05 19:35:56,366::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-05 19:35:56,366::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-05 19:35:56,366::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-05 19:35:56,366::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-05 19:35:56,366::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-05 19:35:56,367::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-05 19:35:58,236::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-05 19:35:58,241::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-05 19:35:58,242::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-05 19:35:58,242::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-05 19:35:58,262::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-05 19:35:58,262::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-05 19:35:58,263::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-05 19:35:58,280::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-05 19:35:58,280::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-05 19:35:58,280::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-05 19:35:58,281::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-05 19:35:58,296::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-05 19:35:58,297::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-05 19:35:58,297::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-05 19:35:58,312::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-05 19:35:58,313::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-05 19:35:58,314::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-05 19:35:58,314::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-05 19:35:58,315::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-05 19:35:58,315::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-05 19:35:58,315::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-05 19:35:58,324::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-05 19:35:58,327::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-05 19:35:58,327::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-05 19:35:58,332::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-05 19:35:58,333::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-05 19:35:58,334::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-05 19:35:58,336::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-05 19:36:00,161::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-05 19:36:00,205::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-05 19:36:00,205::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 19:36:00,206::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 19:36:00,206::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 19:36:00,206::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 19:36:00,206::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-05 19:36:00,206::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-05 19:36:00,207::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-05 19:36:00,207::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-05 19:36:00,210::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-05 19:36:00,210::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-05 19:36:00,211::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-05 19:36:00,211::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-06 00:26:36,420::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-06 00:26:36,421::task::588::TaskManager.Task::(_updateState) Task=`023fe8f4-efbb-4a35-bac8-117420d39920`::moving from state init -> state preparing >MainThread::INFO::2012-12-06 00:26:36,422::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-06 00:26:36,422::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-06 00:26:36,448::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-06 00:26:36,448::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-06 00:26:36,448::task::1172::TaskManager.Task::(prepare) Task=`023fe8f4-efbb-4a35-bac8-117420d39920`::finished: None >MainThread::DEBUG::2012-12-06 00:26:36,449::task::588::TaskManager.Task::(_updateState) Task=`023fe8f4-efbb-4a35-bac8-117420d39920`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-06 00:26:36,449::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-06 00:26:36,449::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-06 00:26:36,449::task::978::TaskManager.Task::(_decref) Task=`023fe8f4-efbb-4a35-bac8-117420d39920`::ref 0 aborting False >MainThread::INFO::2012-12-06 00:26:36,449::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-06 00:26:36,449::vdsm::78::vds::(run) <_MainThread(MainThread, started 139844812003136)> >MainThread::INFO::2012-12-06 00:26:36,449::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139844581566208)> >MainThread::INFO::2012-12-06 00:26:36,450::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139843866720000)> >MainThread::INFO::2012-12-06 00:26:36,450::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139844437128960)> >MainThread::INFO::2012-12-06 00:26:36,450::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139844428736256)> >MainThread::INFO::2012-12-06 00:26:36,450::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139844445521664)> >MainThread::INFO::2012-12-06 00:26:36,450::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139844453914368)> >MainThread::INFO::2012-12-06 00:26:36,450::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139843824756480)> >MainThread::INFO::2012-12-06 00:26:36,450::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139843858327296)> >MainThread::INFO::2012-12-06 00:26:36,450::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-06 00:26:36,450::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139843329849088)> >MainThread::INFO::2012-12-06 00:26:36,451::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139844470699776)> >MainThread::INFO::2012-12-06 00:26:36,451::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139844552197888)> >MainThread::INFO::2012-12-06 00:26:36,451::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139844420343552)> >MainThread::INFO::2012-12-06 00:26:36,451::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139844462307072)> >MainThread::INFO::2012-12-06 00:26:37,617::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-06 00:26:37,782::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-06 00:26:37,782::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-06 00:26:37,802::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 22756' (cwd None) >MainThread::DEBUG::2012-12-06 00:26:37,811::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 22756: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-06 00:26:37,812::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-06 00:26:37,812::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 8db8c836-3263-49fa-9685-abfa34b2e362 31737' (cwd None) >MainThread::DEBUG::2012-12-06 00:26:37,948::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-06 00:26:37,949::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-06 00:26:37,949::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-06 00:26:37,949::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-06 00:26:37,949::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-06 00:26:37,949::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-06 00:26:37,950::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-06 00:26:39,817::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-06 00:26:39,823::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-06 00:26:39,824::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-06 00:26:39,824::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-06 00:26:39,846::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 00:26:39,846::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 00:26:39,847::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-06 00:26:39,864::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 00:26:39,864::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 00:26:39,864::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 00:26:39,865::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-06 00:26:39,880::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-06 00:26:39,881::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 00:26:39,881::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-06 00:26:39,897::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-06 00:26:39,898::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-06 00:26:39,899::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-06 00:26:39,899::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 00:26:39,899::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-06 00:26:39,900::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 00:26:39,900::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-06 00:26:39,908::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-06 00:26:39,912::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-06 00:26:39,913::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-06 00:26:39,921::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-06 00:26:39,922::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-06 00:26:39,923::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-06 00:26:39,925::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-06 00:26:41,747::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-06 00:26:41,787::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-06 00:26:41,787::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 00:26:41,787::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 00:26:41,788::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 00:26:41,788::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 00:26:41,788::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 00:26:41,788::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 00:26:41,788::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-06 00:26:41,789::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-06 00:26:41,792::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-06 00:26:41,792::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-06 00:26:41,792::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-06 00:26:41,793::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-06 05:52:05,510::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-06 05:52:05,510::task::588::TaskManager.Task::(_updateState) Task=`e9b3f00b-4827-4a00-95ac-d6283b13ba34`::moving from state init -> state preparing >MainThread::INFO::2012-12-06 05:52:05,511::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-06 05:52:05,511::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-06 05:52:05,537::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-06 05:52:05,538::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-06 05:52:05,538::task::1172::TaskManager.Task::(prepare) Task=`e9b3f00b-4827-4a00-95ac-d6283b13ba34`::finished: None >MainThread::DEBUG::2012-12-06 05:52:05,538::task::588::TaskManager.Task::(_updateState) Task=`e9b3f00b-4827-4a00-95ac-d6283b13ba34`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-06 05:52:05,538::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-06 05:52:05,538::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-06 05:52:05,538::task::978::TaskManager.Task::(_decref) Task=`e9b3f00b-4827-4a00-95ac-d6283b13ba34`::ref 0 aborting False >MainThread::INFO::2012-12-06 05:52:05,539::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-06 05:52:05,539::vdsm::78::vds::(run) <_MainThread(MainThread, started 139971938793280)> >MainThread::INFO::2012-12-06 05:52:05,539::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139971708356352)> >MainThread::INFO::2012-12-06 05:52:05,539::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139971272881920)> >MainThread::INFO::2012-12-06 05:52:05,539::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139971662202624)> >MainThread::INFO::2012-12-06 05:52:05,539::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139970752796416)> >MainThread::INFO::2012-12-06 05:52:05,539::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139971670595328)> >MainThread::INFO::2012-12-06 05:52:05,540::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139971281274624)> >MainThread::INFO::2012-12-06 05:52:05,540::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139971678988032)> >MainThread::INFO::2012-12-06 05:52:05,540::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-06 05:52:05,540::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139970736011008)> >MainThread::INFO::2012-12-06 05:52:05,540::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139971306452736)> >MainThread::INFO::2012-12-06 05:52:05,540::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139971298060032)> >MainThread::INFO::2012-12-06 05:52:05,540::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139971264489216)> >MainThread::INFO::2012-12-06 05:52:05,540::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139971289667328)> >MainThread::INFO::2012-12-06 05:52:05,541::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139971653809920)> >MainThread::INFO::2012-12-06 05:52:06,663::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-06 05:52:06,826::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-06 05:52:06,827::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-06 05:52:06,845::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 31758' (cwd None) >MainThread::DEBUG::2012-12-06 05:52:06,854::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 31758: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-06 05:52:06,855::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-06 05:52:06,855::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 41774d1e-d211-48b4-bae9-ad11dc98f572 9318' (cwd None) >MainThread::DEBUG::2012-12-06 05:52:06,991::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-06 05:52:06,991::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-06 05:52:06,991::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-06 05:52:06,991::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-06 05:52:06,992::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-06 05:52:06,992::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-06 05:52:06,993::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-06 05:52:08,860::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-06 05:52:08,866::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-06 05:52:08,867::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-06 05:52:08,867::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-06 05:52:08,888::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 05:52:08,888::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 05:52:08,889::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-06 05:52:08,907::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 05:52:08,908::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 05:52:08,908::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 05:52:08,908::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-06 05:52:08,924::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-06 05:52:08,924::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 05:52:08,925::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-06 05:52:08,940::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-06 05:52:08,941::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-06 05:52:08,942::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-06 05:52:08,942::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 05:52:08,942::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-06 05:52:08,942::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 05:52:08,943::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-06 05:52:08,951::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-06 05:52:08,951::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-06 05:52:08,955::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-06 05:52:08,961::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-06 05:52:08,962::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-06 05:52:08,963::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-06 05:52:08,965::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-06 05:52:10,790::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-06 05:52:10,829::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-06 05:52:10,830::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 05:52:10,830::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 05:52:10,830::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 05:52:10,830::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 05:52:10,830::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 05:52:10,831::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 05:52:10,831::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-06 05:52:10,831::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-06 05:52:10,835::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-06 05:52:10,835::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-06 05:52:10,835::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-06 05:52:10,836::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-06 10:29:24,121::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-06 10:29:24,122::task::588::TaskManager.Task::(_updateState) Task=`e9647416-383f-4e30-af2e-03e641cfa96d`::moving from state init -> state preparing >MainThread::INFO::2012-12-06 10:29:24,122::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-06 10:29:24,122::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-06 10:29:24,147::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-06 10:29:24,147::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-06 10:29:24,147::task::1172::TaskManager.Task::(prepare) Task=`e9647416-383f-4e30-af2e-03e641cfa96d`::finished: None >MainThread::DEBUG::2012-12-06 10:29:24,147::task::588::TaskManager.Task::(_updateState) Task=`e9647416-383f-4e30-af2e-03e641cfa96d`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-06 10:29:24,147::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-06 10:29:24,147::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-06 10:29:24,148::task::978::TaskManager.Task::(_decref) Task=`e9647416-383f-4e30-af2e-03e641cfa96d`::ref 0 aborting False >MainThread::INFO::2012-12-06 10:29:24,148::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-06 10:29:24,148::vdsm::78::vds::(run) <_MainThread(MainThread, started 139795985164096)> >MainThread::INFO::2012-12-06 10:29:24,148::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139795754731264)> >MainThread::INFO::2012-12-06 10:29:24,148::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139795321833216)> >MainThread::INFO::2012-12-06 10:29:24,148::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139795347011328)> >MainThread::INFO::2012-12-06 10:29:24,148::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139795313440512)> >MainThread::INFO::2012-12-06 10:29:24,148::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139795338618624)> >MainThread::INFO::2012-12-06 10:29:24,149::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139795330225920)> >MainThread::INFO::2012-12-06 10:29:24,149::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-06 10:29:24,149::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139794717853440)> >MainThread::INFO::2012-12-06 10:29:24,149::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139795691792128)> >MainThread::INFO::2012-12-06 10:29:24,149::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139795700184832)> >MainThread::INFO::2012-12-06 10:29:24,149::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139794734638848)> >MainThread::INFO::2012-12-06 10:29:24,149::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139795725362944)> >MainThread::INFO::2012-12-06 10:29:24,149::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139795716970240)> >MainThread::INFO::2012-12-06 10:29:24,150::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139795708577536)> >MainThread::INFO::2012-12-06 10:29:25,829::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-06 10:29:25,994::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-06 10:29:25,994::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-06 10:29:26,019::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 9339' (cwd None) >MainThread::DEBUG::2012-12-06 10:29:26,028::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 9339: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-06 10:29:26,029::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-06 10:29:26,029::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py a2e761ba-02b2-4642-b2e1-7c2fb964a85a 17584' (cwd None) >MainThread::DEBUG::2012-12-06 10:29:26,163::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-06 10:29:26,163::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-06 10:29:26,163::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-06 10:29:26,163::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-06 10:29:26,163::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-06 10:29:26,164::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-06 10:29:26,165::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-06 10:29:28,033::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-06 10:29:28,038::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-06 10:29:28,039::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-06 10:29:28,039::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-06 10:29:28,059::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 10:29:28,059::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 10:29:28,060::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-06 10:29:28,076::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 10:29:28,076::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 10:29:28,076::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 10:29:28,077::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-06 10:29:28,091::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-06 10:29:28,092::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 10:29:28,092::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-06 10:29:28,106::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-06 10:29:28,107::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-06 10:29:28,108::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-06 10:29:28,108::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 10:29:28,109::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-06 10:29:28,109::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 10:29:28,109::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-06 10:29:28,117::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-06 10:29:28,120::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-06 10:29:28,120::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-06 10:29:28,125::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-06 10:29:28,126::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-06 10:29:28,126::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-06 10:29:28,129::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-06 10:29:29,954::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-06 10:29:29,994::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-06 10:29:29,995::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 10:29:29,995::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 10:29:29,995::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 10:29:29,995::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 10:29:29,996::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 10:29:29,996::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 10:29:29,996::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-06 10:29:29,996::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-06 10:29:30,000::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-06 10:29:30,000::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-06 10:29:30,000::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-06 10:29:30,001::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-06 15:10:20,432::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-06 15:10:20,432::task::588::TaskManager.Task::(_updateState) Task=`30ce3b4c-cf92-4ce9-b17e-6d74f6fa1db1`::moving from state init -> state preparing >MainThread::INFO::2012-12-06 15:10:20,433::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-06 15:10:20,433::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-06 15:10:20,457::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-06 15:10:20,457::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-06 15:10:20,457::task::1172::TaskManager.Task::(prepare) Task=`30ce3b4c-cf92-4ce9-b17e-6d74f6fa1db1`::finished: None >MainThread::DEBUG::2012-12-06 15:10:20,458::task::588::TaskManager.Task::(_updateState) Task=`30ce3b4c-cf92-4ce9-b17e-6d74f6fa1db1`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-06 15:10:20,458::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-06 15:10:20,458::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-06 15:10:20,458::task::978::TaskManager.Task::(_decref) Task=`30ce3b4c-cf92-4ce9-b17e-6d74f6fa1db1`::ref 0 aborting False >MainThread::INFO::2012-12-06 15:10:20,458::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-06 15:10:20,458::vdsm::78::vds::(run) <_MainThread(MainThread, started 140180531439424)> >MainThread::INFO::2012-12-06 15:10:20,458::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140180301002496)> >MainThread::INFO::2012-12-06 15:10:20,458::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140180187166464)> >MainThread::INFO::2012-12-06 15:10:20,459::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140180178773760)> >MainThread::INFO::2012-12-06 15:10:20,459::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140179788515072)> >MainThread::INFO::2012-12-06 15:10:20,459::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140180203951872)> >MainThread::INFO::2012-12-06 15:10:20,459::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-06 15:10:20,459::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140179192928000)> >MainThread::INFO::2012-12-06 15:10:20,459::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140179805300480)> >MainThread::INFO::2012-12-06 15:10:20,459::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140179813693184)> >MainThread::INFO::2012-12-06 15:10:20,459::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140179209713408)> >MainThread::INFO::2012-12-06 15:10:20,460::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140180161988352)> >MainThread::INFO::2012-12-06 15:10:20,460::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140179796907776)> >MainThread::INFO::2012-12-06 15:10:20,460::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140180170381056)> >MainThread::INFO::2012-12-06 15:10:20,460::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140180195559168)> >MainThread::INFO::2012-12-06 15:10:21,731::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-06 15:10:21,895::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-06 15:10:21,896::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-06 15:10:21,915::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 17605' (cwd None) >MainThread::DEBUG::2012-12-06 15:10:21,926::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 17605: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-06 15:10:21,926::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-06 15:10:21,926::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 8faad617-50da-4eb2-8b95-fa11360acd82 25851' (cwd None) >MainThread::DEBUG::2012-12-06 15:10:22,067::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-06 15:10:22,067::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-06 15:10:22,068::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-06 15:10:22,068::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-06 15:10:22,068::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-06 15:10:22,068::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-06 15:10:22,069::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-06 15:10:23,931::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-06 15:10:23,937::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-06 15:10:23,938::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-06 15:10:23,938::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-06 15:10:23,958::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 15:10:23,959::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 15:10:23,960::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-06 15:10:23,976::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 15:10:23,977::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 15:10:23,977::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 15:10:23,977::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-06 15:10:23,993::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-06 15:10:23,993::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 15:10:23,994::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-06 15:10:24,009::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-06 15:10:24,009::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-06 15:10:24,010::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-06 15:10:24,011::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 15:10:24,011::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-06 15:10:24,011::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 15:10:24,012::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-06 15:10:24,020::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-06 15:10:24,023::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-06 15:10:24,024::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-06 15:10:24,032::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-06 15:10:24,033::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-06 15:10:24,034::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-06 15:10:24,037::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-06 15:10:25,859::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-06 15:10:25,905::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-06 15:10:25,905::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 15:10:25,906::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 15:10:25,906::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 15:10:25,906::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 15:10:25,906::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 15:10:25,906::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 15:10:25,907::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-06 15:10:25,907::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-06 15:10:25,911::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-06 15:10:25,911::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-06 15:10:25,911::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-06 15:10:25,911::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-06 20:32:37,316::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-06 20:32:37,317::task::588::TaskManager.Task::(_updateState) Task=`6a6b348f-cd94-4925-82b5-df37e21cb494`::moving from state init -> state preparing >MainThread::INFO::2012-12-06 20:32:37,317::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-06 20:32:37,317::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-06 20:32:37,345::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-06 20:32:37,345::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-06 20:32:37,345::task::1172::TaskManager.Task::(prepare) Task=`6a6b348f-cd94-4925-82b5-df37e21cb494`::finished: None >MainThread::DEBUG::2012-12-06 20:32:37,345::task::588::TaskManager.Task::(_updateState) Task=`6a6b348f-cd94-4925-82b5-df37e21cb494`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-06 20:32:37,345::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-06 20:32:37,345::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-06 20:32:37,346::task::978::TaskManager.Task::(_decref) Task=`6a6b348f-cd94-4925-82b5-df37e21cb494`::ref 0 aborting False >MainThread::INFO::2012-12-06 20:32:37,346::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-06 20:32:37,346::vdsm::78::vds::(run) <_MainThread(MainThread, started 140002642405184)> >MainThread::INFO::2012-12-06 20:32:37,346::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140002411968256)> >MainThread::INFO::2012-12-06 20:32:37,346::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140002277177088)> >MainThread::INFO::2012-12-06 20:32:37,346::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140001757091584)> >MainThread::INFO::2012-12-06 20:32:37,346::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140002285569792)> >MainThread::INFO::2012-12-06 20:32:37,347::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140001773876992)> >MainThread::INFO::2012-12-06 20:32:37,347::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140002293962496)> >MainThread::INFO::2012-12-06 20:32:37,347::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140001765484288)> >MainThread::INFO::2012-12-06 20:32:37,347::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-06 20:32:37,347::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140001228613376)> >MainThread::INFO::2012-12-06 20:32:37,347::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140002310747904)> >MainThread::INFO::2012-12-06 20:32:37,347::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140001723520768)> >MainThread::INFO::2012-12-06 20:32:37,347::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140002268784384)> >MainThread::INFO::2012-12-06 20:32:37,347::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140002260391680)> >MainThread::INFO::2012-12-06 20:32:37,348::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140002302355200)> >MainThread::INFO::2012-12-06 20:32:38,419::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-06 20:32:38,585::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-06 20:32:38,586::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-06 20:32:38,608::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 25872' (cwd None) >MainThread::DEBUG::2012-12-06 20:32:38,618::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 25872: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-06 20:32:38,618::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-06 20:32:38,618::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py b039bad4-4053-4800-9e1e-f85975e0bc4a 3324' (cwd None) >MainThread::DEBUG::2012-12-06 20:32:38,754::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-06 20:32:38,755::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-06 20:32:38,755::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-06 20:32:38,755::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-06 20:32:38,755::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-06 20:32:38,755::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-06 20:32:38,756::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-06 20:32:40,623::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-06 20:32:40,629::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-06 20:32:40,630::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-06 20:32:40,630::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-06 20:32:40,650::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 20:32:40,651::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 20:32:40,652::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-06 20:32:40,669::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-06 20:32:40,669::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 20:32:40,669::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-06 20:32:40,670::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-06 20:32:40,685::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-06 20:32:40,686::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-06 20:32:40,686::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-06 20:32:40,702::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-06 20:32:40,702::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-06 20:32:40,703::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-06 20:32:40,704::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 20:32:40,704::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-06 20:32:40,704::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-06 20:32:40,704::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-06 20:32:40,713::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-06 20:32:40,716::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-06 20:32:40,717::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-06 20:32:40,721::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-06 20:32:40,722::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-06 20:32:40,723::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-06 20:32:40,725::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-06 20:32:42,552::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-06 20:32:42,593::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-06 20:32:42,593::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 20:32:42,593::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 20:32:42,594::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 20:32:42,594::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 20:32:42,594::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-06 20:32:42,594::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-06 20:32:42,594::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-06 20:32:42,595::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-06 20:32:42,598::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-06 20:32:42,598::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-06 20:32:42,598::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-06 20:32:42,599::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-07 02:03:14,522::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-07 02:03:14,523::task::588::TaskManager.Task::(_updateState) Task=`fa8038aa-6dd9-40f2-b5ae-b858454dea15`::moving from state init -> state preparing >MainThread::INFO::2012-12-07 02:03:14,523::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-07 02:03:14,523::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-07 02:03:14,548::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-07 02:03:14,548::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-07 02:03:14,548::task::1172::TaskManager.Task::(prepare) Task=`fa8038aa-6dd9-40f2-b5ae-b858454dea15`::finished: None >MainThread::DEBUG::2012-12-07 02:03:14,548::task::588::TaskManager.Task::(_updateState) Task=`fa8038aa-6dd9-40f2-b5ae-b858454dea15`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-07 02:03:14,548::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-07 02:03:14,549::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-07 02:03:14,549::task::978::TaskManager.Task::(_decref) Task=`fa8038aa-6dd9-40f2-b5ae-b858454dea15`::ref 0 aborting False >MainThread::INFO::2012-12-07 02:03:14,549::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-07 02:03:14,549::vdsm::78::vds::(run) <_MainThread(MainThread, started 140640321857344)> >MainThread::INFO::2012-12-07 02:03:14,549::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140640091420416)> >MainThread::INFO::2012-12-07 02:03:14,549::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140639945602816)> >MainThread::INFO::2012-12-07 02:03:14,549::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140639953995520)> >MainThread::INFO::2012-12-07 02:03:14,550::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140639333230336)> >MainThread::INFO::2012-12-07 02:03:14,550::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140639928817408)> >MainThread::INFO::2012-12-07 02:03:14,550::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140639962388224)> >MainThread::INFO::2012-12-07 02:03:14,550::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-07 02:03:14,550::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140638838322944)> >MainThread::INFO::2012-12-07 02:03:14,550::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140639366801152)> >MainThread::INFO::2012-12-07 02:03:14,550::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140639979173632)> >MainThread::INFO::2012-12-07 02:03:14,550::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140639937210112)> >MainThread::INFO::2012-12-07 02:03:14,551::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140640062052096)> >MainThread::INFO::2012-12-07 02:03:14,551::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140639375193856)> >MainThread::INFO::2012-12-07 02:03:14,551::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140639970780928)> >MainThread::INFO::2012-12-07 02:03:15,948::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-07 02:03:16,111::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-07 02:03:16,111::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-07 02:03:16,136::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 3345' (cwd None) >MainThread::DEBUG::2012-12-07 02:03:16,145::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 3345: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-07 02:03:16,145::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-07 02:03:16,146::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 978ab4ad-40d1-4dd2-bcd3-b2cdd9d32532 13271' (cwd None) >MainThread::DEBUG::2012-12-07 02:03:16,279::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-07 02:03:16,279::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-07 02:03:16,279::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-07 02:03:16,279::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-07 02:03:16,279::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-07 02:03:16,280::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-07 02:03:16,281::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-07 02:03:18,150::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-07 02:03:18,155::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-07 02:03:18,156::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-07 02:03:18,156::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-07 02:03:18,176::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 02:03:18,176::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 02:03:18,177::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-07 02:03:18,193::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 02:03:18,194::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 02:03:18,194::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 02:03:18,194::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-07 02:03:18,209::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-07 02:03:18,210::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 02:03:18,210::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-07 02:03:18,224::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-07 02:03:18,225::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-07 02:03:18,226::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-07 02:03:18,226::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 02:03:18,227::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-07 02:03:18,227::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 02:03:18,227::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-07 02:03:18,235::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-07 02:03:18,238::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-07 02:03:18,238::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-07 02:03:18,243::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-07 02:03:18,243::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-07 02:03:18,244::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-07 02:03:18,246::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-07 02:03:20,072::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-07 02:03:20,116::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-07 02:03:20,116::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 02:03:20,116::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 02:03:20,116::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 02:03:20,117::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 02:03:20,117::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 02:03:20,117::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 02:03:20,117::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-07 02:03:20,117::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-07 02:03:20,121::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-07 02:03:20,121::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-07 02:03:20,121::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-07 02:03:20,122::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-07 07:06:57,207::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-07 07:06:57,207::task::588::TaskManager.Task::(_updateState) Task=`683d07e8-2afe-4e4a-b393-2e866bdccf70`::moving from state init -> state preparing >MainThread::INFO::2012-12-07 07:06:57,208::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-07 07:06:57,208::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-07 07:06:57,232::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-07 07:06:57,233::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-07 07:06:57,233::task::1172::TaskManager.Task::(prepare) Task=`683d07e8-2afe-4e4a-b393-2e866bdccf70`::finished: None >MainThread::DEBUG::2012-12-07 07:06:57,233::task::588::TaskManager.Task::(_updateState) Task=`683d07e8-2afe-4e4a-b393-2e866bdccf70`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-07 07:06:57,233::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-07 07:06:57,233::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-07 07:06:57,233::task::978::TaskManager.Task::(_decref) Task=`683d07e8-2afe-4e4a-b393-2e866bdccf70`::ref 0 aborting False >MainThread::INFO::2012-12-07 07:06:57,233::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-07 07:06:57,234::vdsm::78::vds::(run) <_MainThread(MainThread, started 140278304876352)> >MainThread::INFO::2012-12-07 07:06:57,234::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140278074439424)> >MainThread::INFO::2012-12-07 07:06:57,234::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140277935318784)> >MainThread::INFO::2012-12-07 07:06:57,234::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140277943711488)> >MainThread::INFO::2012-12-07 07:06:57,234::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140277566129920)> >MainThread::INFO::2012-12-07 07:06:57,234::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-07 07:06:57,234::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140276970542848)> >MainThread::INFO::2012-12-07 07:06:57,234::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140277582915328)> >MainThread::INFO::2012-12-07 07:06:57,235::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140277960496896)> >MainThread::INFO::2012-12-07 07:06:57,235::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140277968889600)> >MainThread::INFO::2012-12-07 07:06:57,235::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140277977282304)> >MainThread::INFO::2012-12-07 07:06:57,235::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140277591308032)> >MainThread::INFO::2012-12-07 07:06:57,235::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140277574522624)> >MainThread::INFO::2012-12-07 07:06:57,235::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140277952104192)> >MainThread::INFO::2012-12-07 07:06:57,235::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140276987328256)> >MainThread::INFO::2012-12-07 07:06:58,708::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-07 07:06:58,870::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-07 07:06:58,871::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-07 07:06:58,895::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 13292' (cwd None) >MainThread::DEBUG::2012-12-07 07:06:58,905::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 13292: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-07 07:06:58,906::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-07 07:06:58,906::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py c801c77d-d3b6-44a0-ad33-677bb0c4c890 22466' (cwd None) >MainThread::DEBUG::2012-12-07 07:06:59,043::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-07 07:06:59,043::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-07 07:06:59,043::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-07 07:06:59,043::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-07 07:06:59,043::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-07 07:06:59,044::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-07 07:06:59,045::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-07 07:07:00,911::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-07 07:07:00,917::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-07 07:07:00,918::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-07 07:07:00,918::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-07 07:07:00,938::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 07:07:00,938::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 07:07:00,939::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-07 07:07:00,957::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 07:07:00,957::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 07:07:00,957::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 07:07:00,957::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-07 07:07:00,973::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-07 07:07:00,973::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 07:07:00,974::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-07 07:07:00,989::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-07 07:07:00,990::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-07 07:07:00,991::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-07 07:07:00,991::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 07:07:00,992::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-07 07:07:00,992::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 07:07:00,992::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-07 07:07:01,001::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-07 07:07:01,002::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-07 07:07:01,005::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-07 07:07:01,011::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-07 07:07:01,012::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-07 07:07:01,012::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-07 07:07:01,015::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-07 07:07:02,840::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-07 07:07:02,880::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-07 07:07:02,881::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 07:07:02,881::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 07:07:02,881::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 07:07:02,881::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 07:07:02,882::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 07:07:02,882::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 07:07:02,882::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-07 07:07:02,882::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-07 07:07:02,886::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-07 07:07:02,886::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-07 07:07:02,886::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-07 07:07:02,887::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-07 12:24:31,163::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-07 12:24:31,164::task::588::TaskManager.Task::(_updateState) Task=`e76920e1-bb3a-4315-8f64-99b4ecdc8dbb`::moving from state init -> state preparing >MainThread::INFO::2012-12-07 12:24:31,164::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-07 12:24:31,164::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-07 12:24:31,190::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-07 12:24:31,190::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-07 12:24:31,190::task::1172::TaskManager.Task::(prepare) Task=`e76920e1-bb3a-4315-8f64-99b4ecdc8dbb`::finished: None >MainThread::DEBUG::2012-12-07 12:24:31,190::task::588::TaskManager.Task::(_updateState) Task=`e76920e1-bb3a-4315-8f64-99b4ecdc8dbb`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-07 12:24:31,190::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-07 12:24:31,191::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-07 12:24:31,191::task::978::TaskManager.Task::(_decref) Task=`e76920e1-bb3a-4315-8f64-99b4ecdc8dbb`::ref 0 aborting False >MainThread::INFO::2012-12-07 12:24:31,191::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-07 12:24:31,191::vdsm::78::vds::(run) <_MainThread(MainThread, started 140197677066048)> >MainThread::INFO::2012-12-07 12:24:31,191::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140197446629120)> >MainThread::INFO::2012-12-07 12:24:31,191::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140197161318144)> >MainThread::INFO::2012-12-07 12:24:31,191::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140196607661824)> >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140197169710848)> >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140197178103552)> >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140196658018048)> >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140197152925440)> >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140197194888960)> >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140197144532736)> >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140197408868096)> >MainThread::INFO::2012-12-07 12:24:31,192::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140197417260800)> >MainThread::INFO::2012-12-07 12:24:31,193::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140196624447232)> >MainThread::INFO::2012-12-07 12:24:31,193::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140197186496256)> >MainThread::INFO::2012-12-07 12:24:32,621::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-07 12:24:32,785::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-07 12:24:32,785::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-07 12:24:32,806::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 22487' (cwd None) >MainThread::DEBUG::2012-12-07 12:24:32,817::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 22487: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-07 12:24:32,817::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-07 12:24:32,818::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py bbb53eb9-68ad-4484-8e47-1d9a27ce3474 31638' (cwd None) >MainThread::DEBUG::2012-12-07 12:24:32,953::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-07 12:24:32,954::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-07 12:24:32,954::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-07 12:24:32,954::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-07 12:24:32,954::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-07 12:24:32,954::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-07 12:24:32,955::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-07 12:24:34,823::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-07 12:24:34,828::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-07 12:24:34,829::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-07 12:24:34,829::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-07 12:24:34,850::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 12:24:34,850::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 12:24:34,851::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-07 12:24:34,867::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 12:24:34,868::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 12:24:34,868::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 12:24:34,868::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-07 12:24:34,884::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-07 12:24:34,885::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 12:24:34,885::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-07 12:24:34,900::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-07 12:24:34,901::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-07 12:24:34,902::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-07 12:24:34,902::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 12:24:34,902::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-07 12:24:34,903::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 12:24:34,903::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-07 12:24:34,912::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-07 12:24:34,915::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-07 12:24:34,915::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-07 12:24:34,923::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-07 12:24:34,924::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-07 12:24:34,925::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-07 12:24:34,927::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-07 12:24:36,750::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-07 12:24:36,796::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-07 12:24:36,796::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 12:24:36,797::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 12:24:36,797::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 12:24:36,797::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 12:24:36,797::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 12:24:36,797::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 12:24:36,798::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-07 12:24:36,798::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-07 12:24:36,801::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-07 12:24:36,802::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-07 12:24:36,802::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-07 12:24:36,802::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-07 17:26:01,926::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-07 17:26:01,927::task::588::TaskManager.Task::(_updateState) Task=`2d47d9bd-6da5-41e5-8d66-df446fc8c1ed`::moving from state init -> state preparing >MainThread::INFO::2012-12-07 17:26:01,927::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-07 17:26:01,927::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-07 17:26:01,952::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-07 17:26:01,952::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-07 17:26:01,952::task::1172::TaskManager.Task::(prepare) Task=`2d47d9bd-6da5-41e5-8d66-df446fc8c1ed`::finished: None >MainThread::DEBUG::2012-12-07 17:26:01,952::task::588::TaskManager.Task::(_updateState) Task=`2d47d9bd-6da5-41e5-8d66-df446fc8c1ed`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-07 17:26:01,952::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-07 17:26:01,952::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-07 17:26:01,952::task::978::TaskManager.Task::(_decref) Task=`2d47d9bd-6da5-41e5-8d66-df446fc8c1ed`::ref 0 aborting False >MainThread::INFO::2012-12-07 17:26:01,953::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-07 17:26:01,953::vdsm::78::vds::(run) <_MainThread(MainThread, started 140079753733952)> >MainThread::INFO::2012-12-07 17:26:01,953::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140079523297024)> >MainThread::INFO::2012-12-07 17:26:01,953::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140079385261824)> >MainThread::INFO::2012-12-07 17:26:01,953::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140078831605504)> >MainThread::INFO::2012-12-07 17:26:01,953::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140079393654528)> >MainThread::INFO::2012-12-07 17:26:01,953::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140079402047232)> >MainThread::INFO::2012-12-07 17:26:01,953::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140078873569024)> >MainThread::INFO::2012-12-07 17:26:01,954::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140078881961728)> >MainThread::INFO::2012-12-07 17:26:01,954::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140079418832640)> >MainThread::INFO::2012-12-07 17:26:01,954::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-07 17:26:01,954::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140078336698112)> >MainThread::INFO::2012-12-07 17:26:01,954::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140079376869120)> >MainThread::INFO::2012-12-07 17:26:01,954::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140078865176320)> >MainThread::INFO::2012-12-07 17:26:01,954::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140079368476416)> >MainThread::INFO::2012-12-07 17:26:01,954::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140079410439936)> >MainThread::INFO::2012-12-07 17:26:03,601::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-07 17:26:03,765::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-07 17:26:03,765::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-07 17:26:03,792::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 31659' (cwd None) >MainThread::DEBUG::2012-12-07 17:26:03,801::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 31659: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-07 17:26:03,802::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-07 17:26:03,802::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py c87224c8-135d-4e30-b706-7620a80a931d 8951' (cwd None) >MainThread::DEBUG::2012-12-07 17:26:03,937::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-07 17:26:03,937::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-07 17:26:03,937::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-07 17:26:03,938::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-07 17:26:03,938::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-07 17:26:03,938::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-07 17:26:03,939::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-07 17:26:05,807::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-07 17:26:05,812::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-07 17:26:05,813::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-07 17:26:05,814::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-07 17:26:05,834::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 17:26:05,834::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 17:26:05,835::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-07 17:26:05,852::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 17:26:05,852::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 17:26:05,852::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 17:26:05,853::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-07 17:26:05,869::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-07 17:26:05,869::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 17:26:05,869::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-07 17:26:05,884::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-07 17:26:05,885::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-07 17:26:05,886::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-07 17:26:05,886::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 17:26:05,887::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-07 17:26:05,887::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 17:26:05,887::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-07 17:26:05,896::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-07 17:26:05,899::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-07 17:26:05,900::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-07 17:26:05,904::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-07 17:26:05,905::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-07 17:26:05,906::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-07 17:26:05,908::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-07 17:26:07,734::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-07 17:26:07,780::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-07 17:26:07,781::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 17:26:07,781::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 17:26:07,781::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 17:26:07,781::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 17:26:07,781::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 17:26:07,782::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 17:26:07,782::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-07 17:26:07,782::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-07 17:26:07,786::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-07 17:26:07,786::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-07 17:26:07,786::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-07 17:26:07,787::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-07 22:57:44,893::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-07 22:57:44,894::task::588::TaskManager.Task::(_updateState) Task=`b5f7d7c5-9ba4-4916-8e47-b39ee6ac21b3`::moving from state init -> state preparing >MainThread::INFO::2012-12-07 22:57:44,894::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-07 22:57:44,895::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-07 22:57:44,919::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-07 22:57:44,919::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-07 22:57:44,919::task::1172::TaskManager.Task::(prepare) Task=`b5f7d7c5-9ba4-4916-8e47-b39ee6ac21b3`::finished: None >MainThread::DEBUG::2012-12-07 22:57:44,919::task::588::TaskManager.Task::(_updateState) Task=`b5f7d7c5-9ba4-4916-8e47-b39ee6ac21b3`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-07 22:57:44,920::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-07 22:57:44,920::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-07 22:57:44,920::task::978::TaskManager.Task::(_decref) Task=`b5f7d7c5-9ba4-4916-8e47-b39ee6ac21b3`::ref 0 aborting False >MainThread::INFO::2012-12-07 22:57:44,920::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-07 22:57:44,920::vdsm::78::vds::(run) <_MainThread(MainThread, started 140020173408064)> >MainThread::INFO::2012-12-07 22:57:44,920::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140019942971136)> >MainThread::INFO::2012-12-07 22:57:44,920::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140019121501952)> >MainThread::INFO::2012-12-07 22:57:44,921::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140019658372864)> >MainThread::INFO::2012-12-07 22:57:44,921::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140019913602816)> >MainThread::INFO::2012-12-07 22:57:44,921::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140019666765568)> >MainThread::INFO::2012-12-07 22:57:44,921::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140019675158272)> >MainThread::INFO::2012-12-07 22:57:44,921::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140019905210112)> >MainThread::INFO::2012-12-07 22:57:44,921::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140019691943680)> >MainThread::INFO::2012-12-07 22:57:44,921::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140019155072768)> >MainThread::INFO::2012-12-07 22:57:44,921::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140019649980160)> >MainThread::INFO::2012-12-07 22:57:44,921::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-07 22:57:44,922::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140019104716544)> >MainThread::INFO::2012-12-07 22:57:44,922::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140019641587456)> >MainThread::INFO::2012-12-07 22:57:44,922::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140019683550976)> >MainThread::INFO::2012-12-07 22:57:46,506::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-07 22:57:46,670::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-07 22:57:46,671::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-07 22:57:46,696::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 8972' (cwd None) >MainThread::DEBUG::2012-12-07 22:57:46,705::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 8972: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-07 22:57:46,706::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-07 22:57:46,706::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 2602d72e-880a-4a32-8398-3b85d3d1e17a 18873' (cwd None) >MainThread::DEBUG::2012-12-07 22:57:46,843::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-07 22:57:46,843::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-07 22:57:46,843::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-07 22:57:46,843::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-07 22:57:46,843::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-07 22:57:46,844::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-07 22:57:46,844::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-07 22:57:48,711::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-07 22:57:48,717::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-07 22:57:48,717::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-07 22:57:48,718::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-07 22:57:48,738::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 22:57:48,739::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 22:57:48,740::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-07 22:57:48,756::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-07 22:57:48,756::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 22:57:48,757::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-07 22:57:48,757::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-07 22:57:48,772::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-07 22:57:48,773::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-07 22:57:48,773::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-07 22:57:48,789::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-07 22:57:48,789::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-12::DEBUG::2012-12-07 22:57:48,789::misc::1055::SamplingMethod::(__call__) Got in to sampling method >MainThread::INFO::2012-12-07 22:57:48,790::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-07 22:57:48,791::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-07 22:57:48,791::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-07 22:57:48,791::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-07 22:57:48,800::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-07 22:57:48,800::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-07 22:57:48,804::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-07 22:57:48,809::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-07 22:57:48,810::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-07 22:57:48,810::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-07 22:57:48,813::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-07 22:57:50,639::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-07 22:57:50,687::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-07 22:57:50,687::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 22:57:50,688::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 22:57:50,688::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 22:57:50,688::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 22:57:50,688::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-07 22:57:50,688::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-07 22:57:50,688::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-07 22:57:50,689::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-07 22:57:50,692::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-07 22:57:50,692::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-07 22:57:50,692::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-07 22:57:50,693::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-08 03:46:37,756::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-08 03:46:37,757::task::588::TaskManager.Task::(_updateState) Task=`45ec9a58-1ec0-41ee-8696-81711196e385`::moving from state init -> state preparing >MainThread::INFO::2012-12-08 03:46:37,757::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-08 03:46:37,758::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-08 03:46:37,782::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-08 03:46:37,782::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-08 03:46:37,782::task::1172::TaskManager.Task::(prepare) Task=`45ec9a58-1ec0-41ee-8696-81711196e385`::finished: None >MainThread::DEBUG::2012-12-08 03:46:37,782::task::588::TaskManager.Task::(_updateState) Task=`45ec9a58-1ec0-41ee-8696-81711196e385`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-08 03:46:37,782::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-08 03:46:37,783::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-08 03:46:37,783::task::978::TaskManager.Task::(_decref) Task=`45ec9a58-1ec0-41ee-8696-81711196e385`::ref 0 aborting False >MainThread::INFO::2012-12-08 03:46:37,783::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-08 03:46:37,783::vdsm::78::vds::(run) <_MainThread(MainThread, started 140496406595392)> >MainThread::INFO::2012-12-08 03:46:37,783::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140496176158464)> >MainThread::INFO::2012-12-08 03:46:37,783::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140496070588160)> >MainThread::INFO::2012-12-08 03:46:37,783::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140496078980864)> >MainThread::INFO::2012-12-08 03:46:37,783::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140495669937920)> >MainThread::INFO::2012-12-08 03:46:37,784::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140495678330624)> >MainThread::INFO::2012-12-08 03:46:37,784::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140495661545216)> >MainThread::INFO::2012-12-08 03:46:37,784::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140495686723328)> >MainThread::INFO::2012-12-08 03:46:37,784::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140495149852416)> >MainThread::INFO::2012-12-08 03:46:37,784::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140495695116032)> >MainThread::INFO::2012-12-08 03:46:37,784::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140496045410048)> >MainThread::INFO::2012-12-08 03:46:37,784::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-08 03:46:37,784::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140495133067008)> >MainThread::INFO::2012-12-08 03:46:37,785::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140496053802752)> >MainThread::INFO::2012-12-08 03:46:37,785::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140496062195456)> >MainThread::INFO::2012-12-08 03:46:38,856::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-08 03:46:39,021::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-08 03:46:39,022::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-08 03:46:39,047::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 18894' (cwd None) >MainThread::DEBUG::2012-12-08 03:46:39,057::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 18894: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-08 03:46:39,057::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-08 03:46:39,057::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 638f88e6-1d90-4baf-b009-526df4a9cd77 27856' (cwd None) >MainThread::DEBUG::2012-12-08 03:46:39,192::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-08 03:46:39,192::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-08 03:46:39,192::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-08 03:46:39,193::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-08 03:46:39,193::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-08 03:46:39,193::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-08 03:46:39,194::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-08 03:46:41,063::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-08 03:46:41,068::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-08 03:46:41,069::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-08 03:46:41,069::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-08 03:46:41,089::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-08 03:46:41,090::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-08 03:46:41,090::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-08 03:46:41,107::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-08 03:46:41,108::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-08 03:46:41,108::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-08 03:46:41,108::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-08 03:46:41,124::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-08 03:46:41,124::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-08 03:46:41,124::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-08 03:46:41,140::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-08 03:46:41,140::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-08 03:46:41,141::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-08 03:46:41,142::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-08 03:46:41,142::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-08 03:46:41,142::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-08 03:46:41,143::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-08 03:46:41,151::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-08 03:46:41,154::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-08 03:46:41,155::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-08 03:46:41,159::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-08 03:46:41,160::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-08 03:46:41,161::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-08 03:46:41,163::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-08 03:46:42,989::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-08 03:46:43,037::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-08 03:46:43,037::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 03:46:43,038::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 03:46:43,038::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 03:46:43,038::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 03:46:43,038::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 03:46:43,038::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 03:46:43,039::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-08 03:46:43,039::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-08 03:46:43,042::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-08 03:46:43,042::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-08 03:46:43,043::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-08 03:46:43,043::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-08 09:08:09,716::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-08 09:08:09,717::task::588::TaskManager.Task::(_updateState) Task=`a88d4777-b48a-44db-8cfa-8b28d7a28fc2`::moving from state init -> state preparing >MainThread::INFO::2012-12-08 09:08:09,717::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-08 09:08:09,718::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-08 09:08:09,741::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-08 09:08:09,742::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-08 09:08:09,742::task::1172::TaskManager.Task::(prepare) Task=`a88d4777-b48a-44db-8cfa-8b28d7a28fc2`::finished: None >MainThread::DEBUG::2012-12-08 09:08:09,742::task::588::TaskManager.Task::(_updateState) Task=`a88d4777-b48a-44db-8cfa-8b28d7a28fc2`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-08 09:08:09,742::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-08 09:08:09,742::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-08 09:08:09,742::task::978::TaskManager.Task::(_decref) Task=`a88d4777-b48a-44db-8cfa-8b28d7a28fc2`::ref 0 aborting False >MainThread::INFO::2012-12-08 09:08:09,743::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-08 09:08:09,743::vdsm::78::vds::(run) <_MainThread(MainThread, started 140204528604992)> >MainThread::INFO::2012-12-08 09:08:09,743::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140204298168064)> >MainThread::INFO::2012-12-08 09:08:09,743::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140203410835200)> >MainThread::INFO::2012-12-08 09:08:09,743::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140204260407040)> >MainThread::INFO::2012-12-08 09:08:09,743::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140204252014336)> >MainThread::INFO::2012-12-08 09:08:09,743::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140204268799744)> >MainThread::INFO::2012-12-08 09:08:09,743::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-08 09:08:09,743::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140203394049792)> >MainThread::INFO::2012-12-08 09:08:09,744::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140204023207680)> >MainThread::INFO::2012-12-08 09:08:09,744::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140204031600384)> >MainThread::INFO::2012-12-08 09:08:09,744::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140204006422272)> >MainThread::INFO::2012-12-08 09:08:09,744::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140204014814976)> >MainThread::INFO::2012-12-08 09:08:09,744::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140203998029568)> >MainThread::INFO::2012-12-08 09:08:09,744::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140203989636864)> >MainThread::INFO::2012-12-08 09:08:09,744::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140204039993088)> >MainThread::INFO::2012-12-08 09:08:11,372::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-08 09:08:11,535::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-08 09:08:11,536::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-08 09:08:11,560::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 27877' (cwd None) >MainThread::DEBUG::2012-12-08 09:08:11,569::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 27877: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-08 09:08:11,569::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-08 09:08:11,569::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 9239d4a4-e258-4505-a0d5-3c4df70e9b30 4722' (cwd None) >MainThread::DEBUG::2012-12-08 09:08:11,704::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-08 09:08:11,704::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-08 09:08:11,704::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-08 09:08:11,704::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-08 09:08:11,704::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-08 09:08:11,705::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-08 09:08:11,706::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-08 09:08:13,574::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-08 09:08:13,579::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-08 09:08:13,580::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-08 09:08:13,580::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-08 09:08:13,600::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-08 09:08:13,600::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-08 09:08:13,601::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-08 09:08:13,617::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-08 09:08:13,617::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-08 09:08:13,618::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-08 09:08:13,618::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-08 09:08:13,634::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-08 09:08:13,635::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-08 09:08:13,635::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-08 09:08:13,649::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-08 09:08:13,650::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-08 09:08:13,651::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-08 09:08:13,651::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-08 09:08:13,652::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-08 09:08:13,652::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-08 09:08:13,652::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-08 09:08:13,661::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-08 09:08:13,664::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-08 09:08:13,665::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-08 09:08:13,669::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-08 09:08:13,670::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-08 09:08:13,672::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-08 09:08:13,673::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-08 09:08:15,499::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-08 09:08:15,548::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-08 09:08:15,548::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 09:08:15,549::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 09:08:15,549::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 09:08:15,549::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 09:08:15,549::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 09:08:15,549::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 09:08:15,550::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-08 09:08:15,550::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-08 09:08:15,553::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-08 09:08:15,553::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-08 09:08:15,554::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-08 09:08:15,554::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-08 14:51:15,485::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-08 14:51:15,486::task::588::TaskManager.Task::(_updateState) Task=`92026318-abcc-4742-b2c1-63db4108dfaa`::moving from state init -> state preparing >MainThread::INFO::2012-12-08 14:51:15,486::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-08 14:51:15,486::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-08 14:51:15,512::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-08 14:51:15,513::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-08 14:51:15,513::task::1172::TaskManager.Task::(prepare) Task=`92026318-abcc-4742-b2c1-63db4108dfaa`::finished: None >MainThread::DEBUG::2012-12-08 14:51:15,513::task::588::TaskManager.Task::(_updateState) Task=`92026318-abcc-4742-b2c1-63db4108dfaa`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-08 14:51:15,513::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-08 14:51:15,513::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-08 14:51:15,513::task::978::TaskManager.Task::(_decref) Task=`92026318-abcc-4742-b2c1-63db4108dfaa`::ref 0 aborting False >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::78::vds::(run) <_MainThread(MainThread, started 140160988976960)> >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140160758540032)> >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140159798466304)> >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140160318551808)> >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140160720779008)> >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140160326944512)> >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140160729171712)> >MainThread::INFO::2012-12-08 14:51:15,514::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140160335337216)> >MainThread::INFO::2012-12-08 14:51:15,515::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140160343729920)> >MainThread::INFO::2012-12-08 14:51:15,515::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-08 14:51:15,515::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140159781680896)> >MainThread::INFO::2012-12-08 14:51:15,515::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140160352122624)> >MainThread::INFO::2012-12-08 14:51:15,515::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140160310159104)> >MainThread::INFO::2012-12-08 14:51:15,515::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140160703993600)> >MainThread::INFO::2012-12-08 14:51:15,515::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140160712386304)> >MainThread::INFO::2012-12-08 14:51:17,219::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-08 14:51:17,384::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-08 14:51:17,384::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-08 14:51:17,404::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 4743' (cwd None) >MainThread::DEBUG::2012-12-08 14:51:17,413::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 4743: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-08 14:51:17,414::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-08 14:51:17,414::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 537a76fd-4c51-4fb6-bcc2-1269c8549ee0 15352' (cwd None) >MainThread::DEBUG::2012-12-08 14:51:17,550::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-08 14:51:17,550::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-08 14:51:17,550::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-08 14:51:17,550::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-08 14:51:17,551::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-08 14:51:17,551::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-08 14:51:17,552::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-08 14:51:19,419::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-08 14:51:19,425::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-08 14:51:19,426::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-08 14:51:19,426::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-08 14:51:19,448::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-08 14:51:19,449::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-08 14:51:19,449::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-08 14:51:19,466::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-08 14:51:19,466::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-08 14:51:19,466::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-08 14:51:19,467::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-08 14:51:19,482::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-08 14:51:19,483::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-08 14:51:19,483::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-08 14:51:19,499::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-08 14:51:19,499::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-08 14:51:19,500::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-08 14:51:19,500::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-08 14:51:19,501::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-08 14:51:19,501::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-08 14:51:19,502::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-08 14:51:19,510::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-08 14:51:19,513::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-08 14:51:19,514::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-08 14:51:19,523::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-08 14:51:19,524::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-08 14:51:19,525::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-08 14:51:19,527::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-08 14:51:21,348::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-08 14:51:21,393::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-08 14:51:21,394::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 14:51:21,394::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 14:51:21,394::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 14:51:21,394::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 14:51:21,395::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 14:51:21,395::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 14:51:21,395::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-08 14:51:21,395::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-08 14:51:21,399::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-08 14:51:21,399::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-08 14:51:21,399::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-08 14:51:21,400::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-08 20:16:23,052::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-08 20:16:23,052::task::588::TaskManager.Task::(_updateState) Task=`01d4324f-d831-4306-abde-459d4a561ed6`::moving from state init -> state preparing >MainThread::INFO::2012-12-08 20:16:23,053::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-08 20:16:23,053::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-08 20:16:23,078::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-08 20:16:23,078::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-08 20:16:23,078::task::1172::TaskManager.Task::(prepare) Task=`01d4324f-d831-4306-abde-459d4a561ed6`::finished: None >MainThread::DEBUG::2012-12-08 20:16:23,078::task::588::TaskManager.Task::(_updateState) Task=`01d4324f-d831-4306-abde-459d4a561ed6`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-08 20:16:23,078::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-08 20:16:23,078::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-08 20:16:23,079::task::978::TaskManager.Task::(_decref) Task=`01d4324f-d831-4306-abde-459d4a561ed6`::ref 0 aborting False >MainThread::INFO::2012-12-08 20:16:23,079::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-08 20:16:23,079::vdsm::78::vds::(run) <_MainThread(MainThread, started 140137261696832)> >MainThread::INFO::2012-12-08 20:16:23,079::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140137031259904)> >MainThread::INFO::2012-12-08 20:16:23,079::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140136897558272)> >MainThread::INFO::2012-12-08 20:16:23,079::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-08 20:16:23,079::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140135848994560)> >MainThread::INFO::2012-12-08 20:16:23,079::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140136385865472)> >MainThread::INFO::2012-12-08 20:16:23,080::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140136394258176)> >MainThread::INFO::2012-12-08 20:16:23,080::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140136914343680)> >MainThread::INFO::2012-12-08 20:16:23,080::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140136922736384)> >MainThread::INFO::2012-12-08 20:16:23,080::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140136343901952)> >MainThread::INFO::2012-12-08 20:16:23,080::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140136377472768)> >MainThread::INFO::2012-12-08 20:16:23,080::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140136931129088)> >MainThread::INFO::2012-12-08 20:16:23,080::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140136889165568)> >MainThread::INFO::2012-12-08 20:16:23,080::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140136880772864)> >MainThread::INFO::2012-12-08 20:16:23,081::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140136905950976)> >MainThread::INFO::2012-12-08 20:16:24,235::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-08 20:16:24,397::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-08 20:16:24,398::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-08 20:16:24,419::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 15373' (cwd None) >MainThread::DEBUG::2012-12-08 20:16:24,429::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 15373: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-08 20:16:24,430::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-08 20:16:24,430::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 46d0b531-78d0-4f25-8975-245933c1f7fa 24624' (cwd None) >MainThread::DEBUG::2012-12-08 20:16:24,566::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-08 20:16:24,566::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-08 20:16:24,566::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-08 20:16:24,567::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-08 20:16:24,567::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-08 20:16:24,567::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-08 20:16:24,568::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-08 20:16:26,436::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-08 20:16:26,441::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-08 20:16:26,442::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-08 20:16:26,442::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-08 20:16:26,463::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-08 20:16:26,463::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-08 20:16:26,464::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-08 20:16:26,481::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-08 20:16:26,481::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-08 20:16:26,482::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-08 20:16:26,482::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-08 20:16:26,498::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-08 20:16:26,498::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-08 20:16:26,499::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-08 20:16:26,514::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-08 20:16:26,515::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-08 20:16:26,516::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-08 20:16:26,516::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-08 20:16:26,516::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-08 20:16:26,517::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-08 20:16:26,517::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-08 20:16:26,526::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-08 20:16:26,526::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-08 20:16:26,529::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-08 20:16:26,537::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-08 20:16:26,538::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-08 20:16:26,538::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-08 20:16:26,541::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-08 20:16:28,363::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-08 20:16:28,408::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-08 20:16:28,408::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 20:16:28,409::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 20:16:28,409::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 20:16:28,409::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 20:16:28,409::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-08 20:16:28,409::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-08 20:16:28,410::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-08 20:16:28,410::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-08 20:16:28,414::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-08 20:16:28,414::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-08 20:16:28,414::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-08 20:16:28,414::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-09 01:40:04,078::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-09 01:40:04,079::task::588::TaskManager.Task::(_updateState) Task=`15541a47-23dc-485d-b495-d9ffe6dc4d5c`::moving from state init -> state preparing >MainThread::INFO::2012-12-09 01:40:04,079::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-09 01:40:04,080::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-09 01:40:04,104::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-09 01:40:04,105::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-09 01:40:04,105::task::1172::TaskManager.Task::(prepare) Task=`15541a47-23dc-485d-b495-d9ffe6dc4d5c`::finished: None >MainThread::DEBUG::2012-12-09 01:40:04,105::task::588::TaskManager.Task::(_updateState) Task=`15541a47-23dc-485d-b495-d9ffe6dc4d5c`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-09 01:40:04,105::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-09 01:40:04,105::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-09 01:40:04,105::task::978::TaskManager.Task::(_decref) Task=`15541a47-23dc-485d-b495-d9ffe6dc4d5c`::ref 0 aborting False >MainThread::INFO::2012-12-09 01:40:04,105::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-09 01:40:04,106::vdsm::78::vds::(run) <_MainThread(MainThread, started 139791603132224)> >MainThread::INFO::2012-12-09 01:40:04,106::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139791372695296)> >MainThread::INFO::2012-12-09 01:40:04,106::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139790548711168)> >MainThread::INFO::2012-12-09 01:40:04,106::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139791085582080)> >MainThread::INFO::2012-12-09 01:40:04,106::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139791093974784)> >MainThread::INFO::2012-12-09 01:40:04,106::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-09 01:40:04,106::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139790531925760)> >MainThread::INFO::2012-12-09 01:40:04,106::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139791102367488)> >MainThread::INFO::2012-12-09 01:40:04,107::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139791068796672)> >MainThread::INFO::2012-12-09 01:40:04,107::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139791110760192)> >MainThread::INFO::2012-12-09 01:40:04,107::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139791119152896)> >MainThread::INFO::2012-12-09 01:40:04,107::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139790582281984)> >MainThread::INFO::2012-12-09 01:40:04,107::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139791077189376)> >MainThread::INFO::2012-12-09 01:40:04,107::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139791343326976)> >MainThread::INFO::2012-12-09 01:40:04,107::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139791334934272)> >MainThread::INFO::2012-12-09 01:40:04,922::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-09 01:40:05,085::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-09 01:40:05,086::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-09 01:40:05,111::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 24646' (cwd None) >MainThread::DEBUG::2012-12-09 01:40:05,121::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 24646: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-09 01:40:05,121::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-09 01:40:05,121::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py d61138e8-e0c9-431d-b9b5-bd923d6a174d 2761' (cwd None) >MainThread::DEBUG::2012-12-09 01:40:05,257::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-09 01:40:05,257::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-09 01:40:05,257::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-09 01:40:05,257::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-09 01:40:05,257::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-09 01:40:05,258::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-09 01:40:05,259::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-09 01:40:07,126::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-09 01:40:07,132::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-09 01:40:07,133::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-09 01:40:07,133::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-09 01:40:07,153::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 01:40:07,154::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 01:40:07,155::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-09 01:40:07,172::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 01:40:07,172::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 01:40:07,173::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 01:40:07,173::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-09 01:40:07,189::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-09 01:40:07,189::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 01:40:07,189::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-09 01:40:07,205::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-09 01:40:07,206::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-09 01:40:07,207::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-09 01:40:07,207::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 01:40:07,207::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-09 01:40:07,207::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 01:40:07,208::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-09 01:40:07,216::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-09 01:40:07,219::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-09 01:40:07,220::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-09 01:40:07,224::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-09 01:40:07,225::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-09 01:40:07,227::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-09 01:40:07,228::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-09 01:40:09,067::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-09 01:40:09,106::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-09 01:40:09,106::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 01:40:09,106::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 01:40:09,107::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 01:40:09,107::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 01:40:09,107::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 01:40:09,107::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 01:40:09,107::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-09 01:40:09,108::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-09 01:40:09,111::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-09 01:40:09,111::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-09 01:40:09,111::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-09 01:40:09,112::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-09 07:01:03,255::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-09 07:01:03,256::task::588::TaskManager.Task::(_updateState) Task=`d162db83-01fe-4a8a-96be-65ba5462333a`::moving from state init -> state preparing >MainThread::INFO::2012-12-09 07:01:03,256::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-09 07:01:03,257::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-09 07:01:03,282::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-09 07:01:03,282::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-09 07:01:03,282::task::1172::TaskManager.Task::(prepare) Task=`d162db83-01fe-4a8a-96be-65ba5462333a`::finished: None >MainThread::DEBUG::2012-12-09 07:01:03,282::task::588::TaskManager.Task::(_updateState) Task=`d162db83-01fe-4a8a-96be-65ba5462333a`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-09 07:01:03,282::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-09 07:01:03,282::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-09 07:01:03,283::task::978::TaskManager.Task::(_decref) Task=`d162db83-01fe-4a8a-96be-65ba5462333a`::ref 0 aborting False >MainThread::INFO::2012-12-09 07:01:03,283::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 12 other threads... >MainThread::INFO::2012-12-09 07:01:03,283::vdsm::78::vds::(run) <_MainThread(MainThread, started 140023673616192)> >MainThread::INFO::2012-12-09 07:01:03,283::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140023443179264)> >MainThread::INFO::2012-12-09 07:01:03,283::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140023148033792)> >MainThread::INFO::2012-12-09 07:01:03,283::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140023156426496)> >MainThread::INFO::2012-12-09 07:01:03,283::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140023164819200)> >MainThread::INFO::2012-12-09 07:01:03,283::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140023413810944)> >MainThread::INFO::2012-12-09 07:01:03,284::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140023173211904)> >MainThread::INFO::2012-12-09 07:01:03,284::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140023139641088)> >MainThread::INFO::2012-12-09 07:01:03,284::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140023131248384)> >MainThread::INFO::2012-12-09 07:01:03,284::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140022552446720)> >MainThread::INFO::2012-12-09 07:01:03,284::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140023397025536)> >MainThread::INFO::2012-12-09 07:01:03,284::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140023405418240)> >MainThread::INFO::2012-12-09 07:01:03,284::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140023181604608)> >MainThread::INFO::2012-12-09 07:01:04,788::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-09 07:01:04,953::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-09 07:01:04,953::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-09 07:01:04,978::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 2782' (cwd None) >MainThread::DEBUG::2012-12-09 07:01:04,988::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 2782: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-09 07:01:04,988::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-09 07:01:04,989::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py ae5ff874-a453-4aca-b766-bde526b046cf 12030' (cwd None) >MainThread::DEBUG::2012-12-09 07:01:05,124::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-09 07:01:05,124::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-09 07:01:05,124::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-09 07:01:05,124::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-09 07:01:05,124::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-09 07:01:05,125::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-09 07:01:05,125::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-09 07:01:06,993::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-09 07:01:06,998::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-09 07:01:06,999::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-09 07:01:06,999::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-09 07:01:07,019::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 07:01:07,019::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 07:01:07,020::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-09 07:01:07,037::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 07:01:07,037::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 07:01:07,038::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 07:01:07,038::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-09 07:01:07,053::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-09 07:01:07,053::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 07:01:07,054::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-09 07:01:07,069::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-09 07:01:07,069::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-09 07:01:07,070::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-09 07:01:07,070::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 07:01:07,071::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-09 07:01:07,071::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 07:01:07,072::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-09 07:01:07,079::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-09 07:01:07,082::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-09 07:01:07,082::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-09 07:01:07,087::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-09 07:01:07,089::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-09 07:01:07,089::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-09 07:01:07,092::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-09 07:01:08,917::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-09 07:01:08,966::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-09 07:01:08,966::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 07:01:08,967::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 07:01:08,967::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 07:01:08,967::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 07:01:08,967::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 07:01:08,967::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 07:01:08,967::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-09 07:01:08,968::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-09 07:01:08,971::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-09 07:01:08,971::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-09 07:01:08,971::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-09 07:01:08,972::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-09 13:00:10,041::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-09 13:00:10,042::task::588::TaskManager.Task::(_updateState) Task=`30437dad-d541-44b8-9652-f5d31a4389ca`::moving from state init -> state preparing >MainThread::INFO::2012-12-09 13:00:10,042::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-09 13:00:10,042::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-09 13:00:10,067::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-09 13:00:10,068::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-09 13:00:10,068::task::1172::TaskManager.Task::(prepare) Task=`30437dad-d541-44b8-9652-f5d31a4389ca`::finished: None >MainThread::DEBUG::2012-12-09 13:00:10,068::task::588::TaskManager.Task::(_updateState) Task=`30437dad-d541-44b8-9652-f5d31a4389ca`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-09 13:00:10,068::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-09 13:00:10,068::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-09 13:00:10,068::task::978::TaskManager.Task::(_decref) Task=`30437dad-d541-44b8-9652-f5d31a4389ca`::ref 0 aborting False >MainThread::INFO::2012-12-09 13:00:10,068::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-09 13:00:10,069::vdsm::78::vds::(run) <_MainThread(MainThread, started 140074777274176)> >MainThread::INFO::2012-12-09 13:00:10,069::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140074546837248)> >MainThread::INFO::2012-12-09 13:00:10,069::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140074448893696)> >MainThread::INFO::2012-12-09 13:00:10,069::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140073907513088)> >MainThread::INFO::2012-12-09 13:00:10,069::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140073915905792)> >MainThread::INFO::2012-12-09 13:00:10,069::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140074398537472)> >MainThread::INFO::2012-12-09 13:00:10,069::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-09 13:00:10,069::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140073370642176)> >MainThread::INFO::2012-12-09 13:00:10,070::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140074406930176)> >MainThread::INFO::2012-12-09 13:00:10,070::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140073899120384)> >MainThread::INFO::2012-12-09 13:00:10,070::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140074415322880)> >MainThread::INFO::2012-12-09 13:00:10,070::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140074423715584)> >MainThread::INFO::2012-12-09 13:00:10,070::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140073865549568)> >MainThread::INFO::2012-12-09 13:00:10,070::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140074432108288)> >MainThread::INFO::2012-12-09 13:00:10,070::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140074440500992)> >MainThread::INFO::2012-12-09 13:00:11,086::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-09 13:00:11,251::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-09 13:00:11,251::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-09 13:00:11,272::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 12052' (cwd None) >MainThread::DEBUG::2012-12-09 13:00:11,283::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 12052: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-09 13:00:11,283::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-09 13:00:11,283::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 8e4cb343-1e19-4e0f-9b60-f027f597e360 22806' (cwd None) >MainThread::DEBUG::2012-12-09 13:00:11,419::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-09 13:00:11,419::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-09 13:00:11,419::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-09 13:00:11,420::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-09 13:00:11,420::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-09 13:00:11,420::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-09 13:00:11,421::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-09 13:00:13,288::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-09 13:00:13,294::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-09 13:00:13,295::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-09 13:00:13,295::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-09 13:00:13,317::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 13:00:13,317::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 13:00:13,318::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-09 13:00:13,335::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 13:00:13,336::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 13:00:13,336::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 13:00:13,336::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-09 13:00:13,352::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-09 13:00:13,353::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 13:00:13,353::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-09 13:00:13,369::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-09 13:00:13,369::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-09 13:00:13,370::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-09 13:00:13,371::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 13:00:13,371::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-09 13:00:13,371::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 13:00:13,372::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-09 13:00:13,380::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-09 13:00:13,383::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-09 13:00:13,384::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-09 13:00:13,392::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-09 13:00:13,393::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-09 13:00:13,394::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-09 13:00:13,396::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-09 13:00:15,218::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-09 13:00:15,275::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-09 13:00:15,276::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 13:00:15,276::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 13:00:15,276::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 13:00:15,276::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 13:00:15,276::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 13:00:15,276::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 13:00:15,277::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-09 13:00:15,277::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-09 13:00:15,281::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-09 13:00:15,281::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-09 13:00:15,281::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-09 13:00:15,281::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-09 17:45:05,123::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-09 17:45:05,124::task::588::TaskManager.Task::(_updateState) Task=`63e3f6d6-8fa6-4f97-8b21-ad722d8f3b68`::moving from state init -> state preparing >MainThread::INFO::2012-12-09 17:45:05,124::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-09 17:45:05,124::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-09 17:45:05,150::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-09 17:45:05,150::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-09 17:45:05,150::task::1172::TaskManager.Task::(prepare) Task=`63e3f6d6-8fa6-4f97-8b21-ad722d8f3b68`::finished: None >MainThread::DEBUG::2012-12-09 17:45:05,150::task::588::TaskManager.Task::(_updateState) Task=`63e3f6d6-8fa6-4f97-8b21-ad722d8f3b68`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-09 17:45:05,150::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-09 17:45:05,151::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-09 17:45:05,151::task::978::TaskManager.Task::(_decref) Task=`63e3f6d6-8fa6-4f97-8b21-ad722d8f3b68`::ref 0 aborting False >MainThread::INFO::2012-12-09 17:45:05,151::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-09 17:45:05,151::vdsm::78::vds::(run) <_MainThread(MainThread, started 139991260342080)> >MainThread::INFO::2012-12-09 17:45:05,151::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139991029905152)> >MainThread::INFO::2012-12-09 17:45:05,151::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139990742845184)> >MainThread::INFO::2012-12-09 17:45:05,151::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139990138865408)> >MainThread::INFO::2012-12-09 17:45:05,152::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139990983751424)> >MainThread::INFO::2012-12-09 17:45:05,152::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139990992144128)> >MainThread::INFO::2012-12-09 17:45:05,152::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139991000536832)> >MainThread::INFO::2012-12-09 17:45:05,152::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139990717667072)> >MainThread::INFO::2012-12-09 17:45:05,152::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139990759630592)> >MainThread::INFO::2012-12-09 17:45:05,152::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-09 17:45:05,152::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139990122080000)> >MainThread::INFO::2012-12-09 17:45:05,152::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139990734452480)> >MainThread::INFO::2012-12-09 17:45:05,153::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139990726059776)> >MainThread::INFO::2012-12-09 17:45:05,153::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139990751237888)> >MainThread::INFO::2012-12-09 17:45:05,153::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139990768023296)> >MainThread::INFO::2012-12-09 17:45:06,028::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-09 17:45:06,193::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-09 17:45:06,193::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-09 17:45:06,213::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 22827' (cwd None) >MainThread::DEBUG::2012-12-09 17:45:06,223::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 22827: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-09 17:45:06,224::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-09 17:45:06,224::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py d136d14d-c63d-4c07-be52-5659a687de46 31735' (cwd None) >MainThread::DEBUG::2012-12-09 17:45:06,360::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-09 17:45:06,361::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-09 17:45:06,361::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-09 17:45:06,361::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-09 17:45:06,361::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-09 17:45:06,361::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-09 17:45:06,362::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-09 17:45:08,230::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-09 17:45:08,235::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-09 17:45:08,236::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-09 17:45:08,236::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-09 17:45:08,258::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 17:45:08,259::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 17:45:08,260::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-09 17:45:08,277::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 17:45:08,278::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 17:45:08,278::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 17:45:08,278::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-09 17:45:08,294::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-09 17:45:08,295::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 17:45:08,295::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-09 17:45:08,311::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-09 17:45:08,312::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-09 17:45:08,313::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-09 17:45:08,313::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 17:45:08,313::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-09 17:45:08,314::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 17:45:08,314::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-09 17:45:08,322::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-09 17:45:08,326::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-09 17:45:08,327::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-09 17:45:08,335::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-09 17:45:08,336::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-09 17:45:08,337::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-09 17:45:08,339::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-09 17:45:10,162::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-09 17:45:10,205::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-09 17:45:10,205::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 17:45:10,206::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 17:45:10,206::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 17:45:10,206::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 17:45:10,206::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 17:45:10,206::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 17:45:10,206::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-09 17:45:10,207::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-09 17:45:10,210::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-09 17:45:10,210::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-09 17:45:10,211::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-09 17:45:10,211::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-09 22:34:22,412::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-09 22:34:22,413::task::588::TaskManager.Task::(_updateState) Task=`53302f0e-3f78-427a-a801-11926a7469de`::moving from state init -> state preparing >MainThread::INFO::2012-12-09 22:34:22,413::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-09 22:34:22,413::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-09 22:34:22,437::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-09 22:34:22,437::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-09 22:34:22,437::task::1172::TaskManager.Task::(prepare) Task=`53302f0e-3f78-427a-a801-11926a7469de`::finished: None >MainThread::DEBUG::2012-12-09 22:34:22,437::task::588::TaskManager.Task::(_updateState) Task=`53302f0e-3f78-427a-a801-11926a7469de`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-09 22:34:22,437::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-09 22:34:22,438::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-09 22:34:22,438::task::978::TaskManager.Task::(_decref) Task=`53302f0e-3f78-427a-a801-11926a7469de`::ref 0 aborting False >MainThread::INFO::2012-12-09 22:34:22,438::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-09 22:34:22,438::vdsm::78::vds::(run) <_MainThread(MainThread, started 139705224304448)> >MainThread::INFO::2012-12-09 22:34:22,438::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139704993867520)> >MainThread::INFO::2012-12-09 22:34:22,438::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139704850691840)> >MainThread::INFO::2012-12-09 22:34:22,438::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139704964499200)> >MainThread::INFO::2012-12-09 22:34:22,439::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139704280282880)> >MainThread::INFO::2012-12-09 22:34:22,439::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139704867477248)> >MainThread::INFO::2012-12-09 22:34:22,439::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139704271890176)> >MainThread::INFO::2012-12-09 22:34:22,439::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139704238319360)> >MainThread::INFO::2012-12-09 22:34:22,439::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139704859084544)> >MainThread::INFO::2012-12-09 22:34:22,439::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139704884262656)> >MainThread::INFO::2012-12-09 22:34:22,439::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139704842299136)> >MainThread::INFO::2012-12-09 22:34:22,439::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-09 22:34:22,440::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139703743411968)> >MainThread::INFO::2012-12-09 22:34:22,440::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139704833906432)> >MainThread::INFO::2012-12-09 22:34:22,440::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139704875869952)> >MainThread::INFO::2012-12-09 22:34:23,952::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-09 22:34:24,114::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-09 22:34:24,114::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-09 22:34:24,133::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 31756' (cwd None) >MainThread::DEBUG::2012-12-09 22:34:24,143::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 31756: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-09 22:34:24,143::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-09 22:34:24,144::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py c63b4b02-f83f-4947-814c-7bfcf337ee3a 7704' (cwd None) >MainThread::DEBUG::2012-12-09 22:34:24,280::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-09 22:34:24,280::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-09 22:34:24,280::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-09 22:34:24,281::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-09 22:34:24,281::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-09 22:34:24,281::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-09 22:34:24,282::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-09 22:34:26,149::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-09 22:34:26,155::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-09 22:34:26,156::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-09 22:34:26,156::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-09 22:34:26,178::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 22:34:26,179::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 22:34:26,179::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-09 22:34:26,198::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-09 22:34:26,198::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 22:34:26,198::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-09 22:34:26,199::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-09 22:34:26,215::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-09 22:34:26,215::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-09 22:34:26,216::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-09 22:34:26,232::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-09 22:34:26,233::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-09 22:34:26,234::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-09 22:34:26,234::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 22:34:26,234::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-09 22:34:26,234::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-09 22:34:26,235::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-09 22:34:26,243::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-09 22:34:26,247::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-09 22:34:26,248::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-09 22:34:26,253::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-09 22:34:26,254::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-09 22:34:26,256::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-09 22:34:26,257::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-09 22:34:28,082::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-09 22:34:28,127::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-09 22:34:28,128::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 22:34:28,128::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 22:34:28,128::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 22:34:28,129::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 22:34:28,129::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-09 22:34:28,129::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-09 22:34:28,129::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-09 22:34:28,129::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-09 22:34:28,133::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-09 22:34:28,133::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-09 22:34:28,133::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-09 22:34:28,134::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-10 03:49:25,946::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-10 03:49:25,947::task::588::TaskManager.Task::(_updateState) Task=`c9dcdfcf-cea4-4223-b40c-b8665cb28e14`::moving from state init -> state preparing >MainThread::INFO::2012-12-10 03:49:25,947::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-10 03:49:25,948::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-10 03:49:25,974::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-10 03:49:25,975::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-10 03:49:25,975::task::1172::TaskManager.Task::(prepare) Task=`c9dcdfcf-cea4-4223-b40c-b8665cb28e14`::finished: None >MainThread::DEBUG::2012-12-10 03:49:25,975::task::588::TaskManager.Task::(_updateState) Task=`c9dcdfcf-cea4-4223-b40c-b8665cb28e14`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-10 03:49:25,975::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-10 03:49:25,975::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-10 03:49:25,975::task::978::TaskManager.Task::(_decref) Task=`c9dcdfcf-cea4-4223-b40c-b8665cb28e14`::ref 0 aborting False >MainThread::INFO::2012-12-10 03:49:25,976::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-10 03:49:25,976::vdsm::78::vds::(run) <_MainThread(MainThread, started 140171618879296)> >MainThread::INFO::2012-12-10 03:49:25,976::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140171388442368)> >MainThread::INFO::2012-12-10 03:49:25,976::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140171257296640)> >MainThread::INFO::2012-12-10 03:49:25,976::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-10 03:49:25,976::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140170208732928)> >MainThread::INFO::2012-12-10 03:49:25,976::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140170745603840)> >MainThread::INFO::2012-12-10 03:49:25,976::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140170753996544)> >MainThread::INFO::2012-12-10 03:49:25,977::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140171274082048)> >MainThread::INFO::2012-12-10 03:49:25,977::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140171282474752)> >MainThread::INFO::2012-12-10 03:49:25,977::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140170737211136)> >MainThread::INFO::2012-12-10 03:49:25,977::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140171290867456)> >MainThread::INFO::2012-12-10 03:49:25,977::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140171248903936)> >MainThread::INFO::2012-12-10 03:49:25,977::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140170703640320)> >MainThread::INFO::2012-12-10 03:49:25,977::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140171240511232)> >MainThread::INFO::2012-12-10 03:49:25,977::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140171265689344)> >MainThread::INFO::2012-12-10 03:49:27,181::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-10 03:49:27,346::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-10 03:49:27,347::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-10 03:49:27,368::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 7726' (cwd None) >MainThread::DEBUG::2012-12-10 03:49:27,378::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 7726: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-10 03:49:27,379::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-10 03:49:27,379::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 32158633-c0ad-49b4-93ef-cd295fd8bb73 17514' (cwd None) >MainThread::DEBUG::2012-12-10 03:49:27,516::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-10 03:49:27,517::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-10 03:49:27,517::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-10 03:49:27,517::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-10 03:49:27,517::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-10 03:49:27,517::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-10 03:49:27,518::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-10 03:49:29,384::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-10 03:49:29,390::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-10 03:49:29,391::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-10 03:49:29,391::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-10 03:49:29,412::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 03:49:29,412::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 03:49:29,413::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-10 03:49:29,430::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 03:49:29,430::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 03:49:29,430::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 03:49:29,431::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-10 03:49:29,446::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-10 03:49:29,447::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 03:49:29,447::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-10 03:49:29,463::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-10 03:49:29,463::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-10 03:49:29,464::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-10 03:49:29,465::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 03:49:29,465::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-10 03:49:29,465::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 03:49:29,467::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-10 03:49:29,474::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-10 03:49:29,478::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-10 03:49:29,478::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-10 03:49:29,482::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-10 03:49:29,483::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-10 03:49:29,484::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-10 03:49:29,487::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-10 03:49:31,312::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-10 03:49:31,358::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 03:49:31,359::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 03:49:31,359::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 03:49:31,359::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 03:49:31,359::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 03:49:31,360::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 03:49:31,360::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 03:49:31,360::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-10 03:49:31,360::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-10 03:49:31,364::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-10 03:49:31,364::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-10 03:49:31,364::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-10 03:49:31,365::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-10 09:38:32,760::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-10 09:38:32,761::task::588::TaskManager.Task::(_updateState) Task=`82546acc-7340-41bc-a971-64c4235d6db0`::moving from state init -> state preparing >MainThread::INFO::2012-12-10 09:38:32,761::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-10 09:38:32,762::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-10 09:38:32,788::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-10 09:38:32,788::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-10 09:38:32,788::task::1172::TaskManager.Task::(prepare) Task=`82546acc-7340-41bc-a971-64c4235d6db0`::finished: None >MainThread::DEBUG::2012-12-10 09:38:32,789::task::588::TaskManager.Task::(_updateState) Task=`82546acc-7340-41bc-a971-64c4235d6db0`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-10 09:38:32,789::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-10 09:38:32,789::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-10 09:38:32,789::task::978::TaskManager.Task::(_decref) Task=`82546acc-7340-41bc-a971-64c4235d6db0`::ref 0 aborting False >MainThread::INFO::2012-12-10 09:38:32,789::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-10 09:38:32,789::vdsm::78::vds::(run) <_MainThread(MainThread, started 140505893758784)> >MainThread::INFO::2012-12-10 09:38:32,789::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140505663321856)> >MainThread::INFO::2012-12-10 09:38:32,790::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140505526548224)> >MainThread::INFO::2012-12-10 09:38:32,790::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140505534940928)> >MainThread::INFO::2012-12-10 09:38:32,790::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140504972891904)> >MainThread::INFO::2012-12-10 09:38:32,790::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140505543333632)> >MainThread::INFO::2012-12-10 09:38:32,790::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-10 09:38:32,790::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140504477984512)> >MainThread::INFO::2012-12-10 09:38:32,790::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140505014855424)> >MainThread::INFO::2012-12-10 09:38:32,790::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140505006462720)> >MainThread::INFO::2012-12-10 09:38:32,791::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140505560119040)> >MainThread::INFO::2012-12-10 09:38:32,791::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140505023248128)> >MainThread::INFO::2012-12-10 09:38:32,791::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140505518155520)> >MainThread::INFO::2012-12-10 09:38:32,791::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140505509762816)> >MainThread::INFO::2012-12-10 09:38:32,791::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140505551726336)> >MainThread::INFO::2012-12-10 09:38:34,065::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-10 09:38:34,229::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-10 09:38:34,229::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-10 09:38:34,254::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 17535' (cwd None) >MainThread::DEBUG::2012-12-10 09:38:34,264::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 17535: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-10 09:38:34,264::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-10 09:38:34,264::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 6bc42477-6e1e-4722-8f29-b3bba4137d56 28214' (cwd None) >MainThread::DEBUG::2012-12-10 09:38:34,401::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-10 09:38:34,401::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-10 09:38:34,401::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-10 09:38:34,401::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-10 09:38:34,401::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-10 09:38:34,401::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-10 09:38:34,402::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-10 09:38:36,270::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-10 09:38:36,275::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-10 09:38:36,276::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-10 09:38:36,276::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-10 09:38:36,297::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 09:38:36,297::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 09:38:36,298::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-10 09:38:36,316::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 09:38:36,316::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 09:38:36,316::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 09:38:36,317::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-10 09:38:36,333::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-10 09:38:36,333::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 09:38:36,333::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-10 09:38:36,349::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-10 09:38:36,349::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-10 09:38:36,350::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-10 09:38:36,351::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 09:38:36,351::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-10 09:38:36,351::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 09:38:36,352::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-10 09:38:36,360::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-10 09:38:36,363::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-10 09:38:36,364::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-10 09:38:36,369::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-10 09:38:36,370::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-10 09:38:36,372::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-10 09:38:36,373::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-10 09:38:38,198::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-10 09:38:38,240::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 09:38:38,240::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 09:38:38,240::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 09:38:38,241::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 09:38:38,241::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 09:38:38,241::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 09:38:38,241::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 09:38:38,241::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-10 09:38:38,241::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-10 09:38:38,245::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-10 09:38:38,245::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-10 09:38:38,245::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-10 09:38:38,246::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-10 15:03:48,906::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-10 15:03:48,906::task::588::TaskManager.Task::(_updateState) Task=`faa0b497-810c-4e62-b86b-a718572ff13a`::moving from state init -> state preparing >MainThread::INFO::2012-12-10 15:03:48,907::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-10 15:03:48,907::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-10 15:03:48,932::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-10 15:03:48,932::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-10 15:03:48,932::task::1172::TaskManager.Task::(prepare) Task=`faa0b497-810c-4e62-b86b-a718572ff13a`::finished: None >MainThread::DEBUG::2012-12-10 15:03:48,932::task::588::TaskManager.Task::(_updateState) Task=`faa0b497-810c-4e62-b86b-a718572ff13a`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-10 15:03:48,932::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-10 15:03:48,932::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-10 15:03:48,932::task::978::TaskManager.Task::(_decref) Task=`faa0b497-810c-4e62-b86b-a718572ff13a`::ref 0 aborting False >MainThread::INFO::2012-12-10 15:03:48,933::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 12 other threads... >MainThread::INFO::2012-12-10 15:03:48,933::vdsm::78::vds::(run) <_MainThread(MainThread, started 140072926226240)> >MainThread::INFO::2012-12-10 15:03:48,933::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140072695789312)> >MainThread::INFO::2012-12-10 15:03:48,933::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140072649635584)> >MainThread::INFO::2012-12-10 15:03:48,933::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140072658028288)> >MainThread::INFO::2012-12-10 15:03:48,933::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140072414332672)> >MainThread::INFO::2012-12-10 15:03:48,933::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140072666420992)> >MainThread::INFO::2012-12-10 15:03:48,933::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140072389154560)> >MainThread::INFO::2012-12-10 15:03:48,934::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140072431118080)> >MainThread::INFO::2012-12-10 15:03:48,934::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140071810352896)> >MainThread::INFO::2012-12-10 15:03:48,934::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140072405939968)> >MainThread::INFO::2012-12-10 15:03:48,934::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140072397547264)> >MainThread::INFO::2012-12-10 15:03:48,934::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140072422725376)> >MainThread::INFO::2012-12-10 15:03:48,934::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140072439510784)> >MainThread::INFO::2012-12-10 15:03:50,242::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-10 15:03:50,405::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-10 15:03:50,405::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-10 15:03:50,430::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 28235' (cwd None) >MainThread::DEBUG::2012-12-10 15:03:50,439::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 28235: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-10 15:03:50,440::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-10 15:03:50,440::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py e32e939e-aa77-46bd-b69a-07515c1d2008 5128' (cwd None) >MainThread::DEBUG::2012-12-10 15:03:50,575::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-10 15:03:50,575::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-10 15:03:50,575::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-10 15:03:50,576::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-10 15:03:50,576::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-10 15:03:50,576::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-10 15:03:50,577::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-10 15:03:52,444::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-10 15:03:52,450::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-10 15:03:52,451::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-10 15:03:52,451::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-10 15:03:52,471::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 15:03:52,471::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 15:03:52,472::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-10 15:03:52,489::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 15:03:52,490::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 15:03:52,490::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 15:03:52,490::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-10 15:03:52,505::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-10 15:03:52,506::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 15:03:52,506::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-10 15:03:52,521::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-10 15:03:52,521::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-10 15:03:52,522::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-10 15:03:52,523::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 15:03:52,523::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-10 15:03:52,523::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 15:03:52,524::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-10 15:03:52,532::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-10 15:03:52,534::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-10 15:03:52,535::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-10 15:03:52,540::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-10 15:03:52,541::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-10 15:03:52,541::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-10 15:03:52,544::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-10 15:03:54,369::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-10 15:03:54,418::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 15:03:54,419::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 15:03:54,419::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 15:03:54,419::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 15:03:54,419::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 15:03:54,419::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 15:03:54,420::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 15:03:54,420::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-10 15:03:54,420::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-10 15:03:54,424::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-10 15:03:54,424::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-10 15:03:54,424::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-10 15:03:54,425::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-10 15:27:37,254::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-10 15:27:37,255::task::588::TaskManager.Task::(_updateState) Task=`1b5bd284-58f7-45f0-ad8b-d866b61262d2`::moving from state init -> state preparing >MainThread::INFO::2012-12-10 15:27:37,255::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-10 15:27:37,255::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-10 15:27:37,273::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-10 15:27:37,273::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-10 15:27:37,274::task::1172::TaskManager.Task::(prepare) Task=`1b5bd284-58f7-45f0-ad8b-d866b61262d2`::finished: None >MainThread::DEBUG::2012-12-10 15:27:37,274::task::588::TaskManager.Task::(_updateState) Task=`1b5bd284-58f7-45f0-ad8b-d866b61262d2`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-10 15:27:37,274::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-10 15:27:37,274::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-10 15:27:37,274::task::978::TaskManager.Task::(_decref) Task=`1b5bd284-58f7-45f0-ad8b-d866b61262d2`::ref 0 aborting False >MainThread::INFO::2012-12-10 15:27:37,274::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-10 15:27:37,274::vdsm::78::vds::(run) <_MainThread(MainThread, started 140380792244032)> >MainThread::INFO::2012-12-10 15:27:37,275::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140380561807104)> >MainThread::INFO::2012-12-10 15:27:37,275::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140380465313536)> >MainThread::INFO::2012-12-10 15:27:37,275::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140379932325632)> >MainThread::INFO::2012-12-10 15:27:37,275::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140379923932928)> >MainThread::INFO::2012-12-10 15:27:37,275::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-10 15:27:37,275::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140379387062016)> >MainThread::INFO::2012-12-10 15:27:37,275::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140380414957312)> >MainThread::INFO::2012-12-10 15:27:37,275::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140379915540224)> >MainThread::INFO::2012-12-10 15:27:37,276::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140380423350016)> >MainThread::INFO::2012-12-10 15:27:37,276::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140380431742720)> >MainThread::INFO::2012-12-10 15:27:37,276::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140380440135424)> >MainThread::INFO::2012-12-10 15:27:37,276::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140379881969408)> >MainThread::INFO::2012-12-10 15:27:37,276::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140380448528128)> >MainThread::INFO::2012-12-10 15:27:37,276::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140380456920832)> >MainThread::INFO::2012-12-10 17:29:28,014::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-10 17:29:29,781::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-10 17:29:29,781::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-10 17:29:29,842::supervdsm::103::SuperVdsmProxy::(_killSupervdsm) Could not kill old Super Vdsm [Errno 2] No such file or directory: '/var/run/vdsm/svdsm.pid' >MainThread::DEBUG::2012-12-10 17:29:29,842::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-10 17:29:29,843::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 679ac0a4-8ead-42e9-802a-bde6deb95639 1254' (cwd None) >MainThread::DEBUG::2012-12-10 17:29:29,984::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-10 17:29:29,984::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-10 17:29:29,984::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-10 17:29:29,984::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-10 17:29:29,984::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-10 17:29:29,985::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-10 17:29:29,985::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-10 17:29:31,849::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-10 17:29:31,854::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-10 17:29:31,855::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-10 17:29:31,855::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-10 17:29:31,877::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:29:31,877::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 17:29:31,878::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-10 17:29:31,895::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:29:31,895::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 17:29:31,895::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 17:29:31,896::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-10 17:29:31,911::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:29:31,912::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 17:29:31,912::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-10 17:29:31,927::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:29:31,928::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-10 17:29:31,929::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-10 17:29:31,929::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 17:29:31,930::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-10 17:29:31,930::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 17:29:31,930::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-10 17:29:31,938::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-10 17:29:31,942::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21 >Thread-12::DEBUG::2012-12-10 17:29:31,943::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-10 17:29:31,950::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-10 17:29:31,951::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-10 17:29:31,951::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-10 17:29:31,954::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-10 17:29:33,777::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-10 17:29:33,823::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:29:33,824::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:29:33,824::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:29:33,824::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:29:33,824::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:29:33,824::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:29:33,825::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:29:33,825::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-10 17:29:33,825::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-10 17:29:33,829::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-10 17:29:33,829::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-10 17:29:33,829::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-10 17:29:33,830::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-10 16:51:34,031::BindingXMLRPC::859::vds::(wrapper) client [10.35.148.46]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-10 16:51:34,203::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-14::DEBUG::2012-12-10 16:51:34,219::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 16:51:34,219::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-10 16:51:34,234::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 16:51:34,234::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-10 16:51:34,249::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 16:51:34,250::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-10 16:51:34,264::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 16:51:34,264::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-14::DEBUG::2012-12-10 16:51:34,279::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 16:51:35,055::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >Thread-15::DEBUG::2012-12-10 17:13:50,035::BindingXMLRPC::859::vds::(wrapper) client [10.35.148.46]::call getCapabilities with () {} >Thread-15::DEBUG::2012-12-10 17:13:50,054::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-15::DEBUG::2012-12-10 17:13:50,076::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:13:50,076::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-15::DEBUG::2012-12-10 17:13:50,092::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:13:50,092::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-15::DEBUG::2012-12-10 17:13:50,108::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:13:50,108::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-15::DEBUG::2012-12-10 17:13:50,122::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:13:50,123::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-15::DEBUG::2012-12-10 17:13:50,138::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:13:50,138::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '10.35.148.46'}} >MainThread::INFO::2012-12-10 17:14:18,055::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-10 17:14:18,056::task::588::TaskManager.Task::(_updateState) Task=`90c91ed0-0182-4c37-99c7-b4fe39b8a296`::moving from state init -> state preparing >MainThread::INFO::2012-12-10 17:14:18,056::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-10 17:14:18,056::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-10 17:14:18,086::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-10 17:14:18,086::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-10 17:14:18,086::task::1172::TaskManager.Task::(prepare) Task=`90c91ed0-0182-4c37-99c7-b4fe39b8a296`::finished: None >MainThread::DEBUG::2012-12-10 17:14:18,087::task::588::TaskManager.Task::(_updateState) Task=`90c91ed0-0182-4c37-99c7-b4fe39b8a296`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-10 17:14:18,087::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-10 17:14:18,087::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-10 17:14:18,087::task::978::TaskManager.Task::(_decref) Task=`90c91ed0-0182-4c37-99c7-b4fe39b8a296`::ref 0 aborting False >MainThread::INFO::2012-12-10 17:14:18,087::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-10 17:14:18,087::vdsm::78::vds::(run) <_MainThread(MainThread, started 139903885395776)> >MainThread::INFO::2012-12-10 17:14:18,087::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139903654958848)> >MainThread::INFO::2012-12-10 17:14:18,088::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139903358711552)> >MainThread::INFO::2012-12-10 17:14:18,088::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139903383889664)> >MainThread::INFO::2012-12-10 17:14:18,088::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139902763124480)> >MainThread::INFO::2012-12-10 17:14:18,088::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139903608805120)> >MainThread::INFO::2012-12-10 17:14:18,088::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-10 17:14:18,088::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139902746339072)> >MainThread::INFO::2012-12-10 17:14:18,088::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139903617197824)> >MainThread::INFO::2012-12-10 17:14:18,088::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139903625590528)> >MainThread::INFO::2012-12-10 17:14:18,088::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139903367104256)> >MainThread::INFO::2012-12-10 17:14:18,089::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139903350318848)> >MainThread::INFO::2012-12-10 17:14:18,089::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139903375496960)> >MainThread::INFO::2012-12-10 17:14:18,089::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139903341926144)> >MainThread::INFO::2012-12-10 17:14:18,089::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139903392282368)> >MainThread::INFO::2012-12-10 19:16:37,441::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-10 19:16:39,080::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-10 19:16:39,081::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-10 19:16:39,143::supervdsm::103::SuperVdsmProxy::(_killSupervdsm) Could not kill old Super Vdsm [Errno 2] No such file or directory: '/var/run/vdsm/svdsm.pid' >MainThread::DEBUG::2012-12-10 19:16:39,143::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-10 19:16:39,143::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py aae5941e-1357-43ab-a0cf-502f6139545d 1254' (cwd None) >MainThread::DEBUG::2012-12-10 19:16:39,281::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-10 19:16:39,281::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-10 19:16:39,281::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-10 19:16:39,282::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-10 19:16:39,282::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-10 19:16:39,282::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-10 19:16:39,283::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-10 19:16:41,148::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-10 19:16:41,153::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-10 19:16:41,154::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-10 19:16:41,154::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-10 19:16:41,174::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 19:16:41,175::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 19:16:41,176::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-10 19:16:41,192::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 19:16:41,192::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 19:16:41,192::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 19:16:41,193::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-10 19:16:41,207::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-10 19:16:41,207::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 19:16:41,207::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-10 19:16:41,222::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-10 19:16:41,222::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-10 19:16:41,223::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-10 19:16:41,223::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 19:16:41,224::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-10 19:16:41,224::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 19:16:41,224::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-10 19:16:41,232::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-10 19:16:41,235::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 19:16:41,235::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-10 19:16:41,243::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-10 19:16:41,245::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-10 19:16:41,245::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-10 19:16:41,247::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-10 19:16:43,070::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-10 19:16:43,108::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 19:16:43,108::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 19:16:43,108::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 19:16:43,109::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 19:16:43,109::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 19:16:43,109::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 19:16:43,109::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 19:16:43,109::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-10 19:16:43,110::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-10 19:16:43,113::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-10 19:16:43,113::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-10 19:16:43,113::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-10 19:16:43,114::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-10 17:16:57,199::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-10 17:16:57,362::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-14::DEBUG::2012-12-10 17:16:57,377::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 17:16:57,378::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-10 17:16:57,392::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 17:16:57,392::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-10 17:16:57,406::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 17:16:57,407::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-10 17:16:57,420::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 17:16:57,421::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-14::DEBUG::2012-12-10 17:16:57,435::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-10 17:16:58,213::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >Thread-15::DEBUG::2012-12-10 17:16:58,482::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-15::DEBUG::2012-12-10 17:16:58,496::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-15::DEBUG::2012-12-10 17:16:58,511::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:16:58,511::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-15::DEBUG::2012-12-10 17:16:58,525::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:16:58,526::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-15::DEBUG::2012-12-10 17:16:58,540::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:16:58,540::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-15::DEBUG::2012-12-10 17:16:58,554::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:16:58,554::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-15::DEBUG::2012-12-10 17:16:58,568::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-10 17:16:58,569::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '3.fc18', 'buildtime': '1352815746', 'version': '0.10.2.1'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '10.35.18.155'}} >Thread-19::DEBUG::2012-12-10 17:17:05,143::task::588::TaskManager.Task::(_updateState) Task=`970840fc-c232-4048-b06b-26b372b4c387`::moving from state init -> state preparing >Thread-19::INFO::2012-12-10 17:17:05,144::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-19::INFO::2012-12-10 17:17:05,144::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-19::DEBUG::2012-12-10 17:17:05,147::task::1172::TaskManager.Task::(prepare) Task=`970840fc-c232-4048-b06b-26b372b4c387`::finished: {} >Thread-19::DEBUG::2012-12-10 17:17:05,147::task::588::TaskManager.Task::(_updateState) Task=`970840fc-c232-4048-b06b-26b372b4c387`::moving from state preparing -> state finished >Thread-19::DEBUG::2012-12-10 17:17:05,147::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-19::DEBUG::2012-12-10 17:17:05,148::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-19::DEBUG::2012-12-10 17:17:05,148::task::978::TaskManager.Task::(_decref) Task=`970840fc-c232-4048-b06b-26b372b4c387`::ref 0 aborting False >Thread-23::DEBUG::2012-12-10 17:17:10,826::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-23::DEBUG::2012-12-10 17:17:10,826::task::588::TaskManager.Task::(_updateState) Task=`aa773f92-a115-4032-817c-c59a700b5bf7`::moving from state init -> state preparing >Thread-23::INFO::2012-12-10 17:17:10,826::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-23::INFO::2012-12-10 17:17:10,827::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-23::DEBUG::2012-12-10 17:17:10,827::task::1172::TaskManager.Task::(prepare) Task=`aa773f92-a115-4032-817c-c59a700b5bf7`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-23::DEBUG::2012-12-10 17:17:10,827::task::588::TaskManager.Task::(_updateState) Task=`aa773f92-a115-4032-817c-c59a700b5bf7`::moving from state preparing -> state finished >Thread-23::DEBUG::2012-12-10 17:17:10,827::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-23::DEBUG::2012-12-10 17:17:10,827::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-23::DEBUG::2012-12-10 17:17:10,827::task::978::TaskManager.Task::(_decref) Task=`aa773f92-a115-4032-817c-c59a700b5bf7`::ref 0 aborting False >Thread-24::DEBUG::2012-12-10 17:17:10,931::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-24::DEBUG::2012-12-10 17:17:10,931::task::588::TaskManager.Task::(_updateState) Task=`53797e0d-67f3-43ee-9632-830e3bf7db39`::moving from state init -> state preparing >Thread-24::INFO::2012-12-10 17:17:10,931::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-24::DEBUG::2012-12-10 17:17:10,953::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386 /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-24::DEBUG::2012-12-10 17:17:11,209::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-24::DEBUG::2012-12-10 17:17:11,209::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-24::DEBUG::2012-12-10 17:17:11,209::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-24::DEBUG::2012-12-10 17:17:11,210::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-24::DEBUG::2012-12-10 17:17:11,210::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-24::DEBUG::2012-12-10 17:17:11,210::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-24::INFO::2012-12-10 17:17:11,210::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-24::DEBUG::2012-12-10 17:17:11,210::task::1172::TaskManager.Task::(prepare) Task=`53797e0d-67f3-43ee-9632-830e3bf7db39`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-24::DEBUG::2012-12-10 17:17:11,211::task::588::TaskManager.Task::(_updateState) Task=`53797e0d-67f3-43ee-9632-830e3bf7db39`::moving from state preparing -> state finished >Thread-24::DEBUG::2012-12-10 17:17:11,211::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-24::DEBUG::2012-12-10 17:17:11,211::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-24::DEBUG::2012-12-10 17:17:11,211::task::978::TaskManager.Task::(_decref) Task=`53797e0d-67f3-43ee-9632-830e3bf7db39`::ref 0 aborting False >Thread-25::DEBUG::2012-12-10 17:17:11,367::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-25::DEBUG::2012-12-10 17:17:11,367::task::588::TaskManager.Task::(_updateState) Task=`14069746-222d-42ab-bdcb-ba3b07da3b9a`::moving from state init -> state preparing >Thread-25::INFO::2012-12-10 17:17:11,367::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=1, storageType=1, remotePath='10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', options=None) >Thread-25::DEBUG::2012-12-10 17:17:11,367::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-25::DEBUG::2012-12-10 17:17:11,367::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-25::DEBUG::2012-12-10 17:17:11,368::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-25::DEBUG::2012-12-10 17:17:11,368::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-25::DEBUG::2012-12-10 17:17:11,368::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-25::DEBUG::2012-12-10 17:17:11,378::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-10 17:17:11,379::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-25::DEBUG::2012-12-10 17:17:13,213::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-25::DEBUG::2012-12-10 17:17:13,252::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-10 17:17:13,252::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-25::DEBUG::2012-12-10 17:17:13,253::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-25::DEBUG::2012-12-10 17:17:13,253::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-25::DEBUG::2012-12-10 17:17:13,253::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-25::DEBUG::2012-12-10 17:17:13,253::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-25::DEBUG::2012-12-10 17:17:13,253::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-25::DEBUG::2012-12-10 17:17:13,254::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-25::DEBUG::2012-12-10 17:17:13,254::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-25::DEBUG::2012-12-10 17:17:13,254::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-25::DEBUG::2012-12-10 17:17:13,277::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-25::DEBUG::2012-12-10 17:17:13,277::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-25::INFO::2012-12-10 17:17:13,447::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-25::DEBUG::2012-12-10 17:17:13,447::task::1172::TaskManager.Task::(prepare) Task=`14069746-222d-42ab-bdcb-ba3b07da3b9a`::finished: {'domlist': []} >Thread-25::DEBUG::2012-12-10 17:17:13,447::task::588::TaskManager.Task::(_updateState) Task=`14069746-222d-42ab-bdcb-ba3b07da3b9a`::moving from state preparing -> state finished >Thread-25::DEBUG::2012-12-10 17:17:13,448::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-25::DEBUG::2012-12-10 17:17:13,448::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-25::DEBUG::2012-12-10 17:17:13,448::task::978::TaskManager.Task::(_decref) Task=`14069746-222d-42ab-bdcb-ba3b07da3b9a`::ref 0 aborting False >Thread-29::DEBUG::2012-12-10 17:17:13,697::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-29::DEBUG::2012-12-10 17:17:13,697::task::588::TaskManager.Task::(_updateState) Task=`4f64ae02-3219-4bb4-a9e5-d59c36bb1b19`::moving from state init -> state preparing >Thread-29::INFO::2012-12-10 17:17:13,697::logUtils::37::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='5838074b-e545-49d3-a4c8-da1ecd091999', domainName='nfsStorageTest_data_domain_0', typeSpecificArg='10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', domClass=1, domVersion='3', options=None) >Thread-29::DEBUG::2012-12-10 17:17:13,698::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-29::DEBUG::2012-12-10 17:17:13,698::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 5838074b-e545-49d3-a4c8-da1ecd091999' (cwd None) >Thread-29::DEBUG::2012-12-10 17:17:13,716::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "5838074b-e545-49d3-a4c8-da1ecd091999" not found\n'; <rc> = 5 >Thread-29::WARNING::2012-12-10 17:17:13,717::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "5838074b-e545-49d3-a4c8-da1ecd091999" not found'] >Thread-29::DEBUG::2012-12-10 17:17:13,717::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-29::INFO::2012-12-10 17:17:13,720::nfsSD::64::Storage.StorageDomain::(create) sdUUID=5838074b-e545-49d3-a4c8-da1ecd091999 domainName=nfsStorageTest_data_domain_0 remotePath=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386 domClass=1 >Thread-29::DEBUG::2012-12-10 17:17:13,733::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-29::DEBUG::2012-12-10 17:17:13,734::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-29::WARNING::2012-12-10 17:17:13,734::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-29::DEBUG::2012-12-10 17:17:13,734::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-29::DEBUG::2012-12-10 17:17:13,734::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-29::DEBUG::2012-12-10 17:17:13,735::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f4a6536b639f001552cb32b51970faa9bb89bca5'] >Thread-29::DEBUG::2012-12-10 17:17:13,745::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-29::DEBUG::2012-12-10 17:17:13,746::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-29::DEBUG::2012-12-10 17:17:13,747::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-29::DEBUG::2012-12-10 17:17:13,749::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f4a6536b639f001552cb32b51970faa9bb89bca5'] >Thread-29::DEBUG::2012-12-10 17:17:13,751::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-29::DEBUG::2012-12-10 17:17:13,751::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '5838074b-e545-49d3-a4c8-da1ecd091999_imageNS' >Thread-29::DEBUG::2012-12-10 17:17:13,751::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '5838074b-e545-49d3-a4c8-da1ecd091999_volumeNS' >Thread-29::DEBUG::2012-12-10 17:17:13,773::sd::354::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-29::INFO::2012-12-10 17:17:13,774::logUtils::39::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-29::DEBUG::2012-12-10 17:17:13,774::task::1172::TaskManager.Task::(prepare) Task=`4f64ae02-3219-4bb4-a9e5-d59c36bb1b19`::finished: None >Thread-29::DEBUG::2012-12-10 17:17:13,774::task::588::TaskManager.Task::(_updateState) Task=`4f64ae02-3219-4bb4-a9e5-d59c36bb1b19`::moving from state preparing -> state finished >Thread-29::DEBUG::2012-12-10 17:17:13,774::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-29::DEBUG::2012-12-10 17:17:13,774::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-29::DEBUG::2012-12-10 17:17:13,774::task::978::TaskManager.Task::(_decref) Task=`4f64ae02-3219-4bb4-a9e5-d59c36bb1b19`::ref 0 aborting False >Thread-32::DEBUG::2012-12-10 17:17:13,853::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-32::DEBUG::2012-12-10 17:17:13,853::task::588::TaskManager.Task::(_updateState) Task=`ae62b0f8-3dbe-4be4-8a70-daeaa2827f73`::moving from state init -> state preparing >Thread-32::INFO::2012-12-10 17:17:13,853::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='5838074b-e545-49d3-a4c8-da1ecd091999', options=None) >Thread-32::DEBUG::2012-12-10 17:17:13,854::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.5838074b-e545-49d3-a4c8-da1ecd091999`ReqID=`ac0ed82d-368b-40dc-a283-c2dde4316051`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-32::DEBUG::2012-12-10 17:17:13,854::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' for lock type 'shared' >Thread-32::DEBUG::2012-12-10 17:17:13,854::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' is free. Now locking as 'shared' (1 active user) >Thread-32::DEBUG::2012-12-10 17:17:13,854::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.5838074b-e545-49d3-a4c8-da1ecd091999`ReqID=`ac0ed82d-368b-40dc-a283-c2dde4316051`::Granted request >Thread-32::DEBUG::2012-12-10 17:17:13,855::task::817::TaskManager.Task::(resourceAcquired) Task=`ae62b0f8-3dbe-4be4-8a70-daeaa2827f73`::_resourcesAcquired: Storage.5838074b-e545-49d3-a4c8-da1ecd091999 (shared) >Thread-32::DEBUG::2012-12-10 17:17:13,855::task::978::TaskManager.Task::(_decref) Task=`ae62b0f8-3dbe-4be4-8a70-daeaa2827f73`::ref 1 aborting False >Thread-32::DEBUG::2012-12-10 17:17:13,855::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-32::DEBUG::2012-12-10 17:17:13,855::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 5838074b-e545-49d3-a4c8-da1ecd091999' (cwd None) >Thread-32::DEBUG::2012-12-10 17:17:13,872::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "5838074b-e545-49d3-a4c8-da1ecd091999" not found\n'; <rc> = 5 >Thread-32::WARNING::2012-12-10 17:17:13,872::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "5838074b-e545-49d3-a4c8-da1ecd091999" not found'] >Thread-32::DEBUG::2012-12-10 17:17:13,872::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-32::DEBUG::2012-12-10 17:17:13,877::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-32::DEBUG::2012-12-10 17:17:13,878::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-32::DEBUG::2012-12-10 17:17:13,879::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f4a6536b639f001552cb32b51970faa9bb89bca5'] >Thread-32::DEBUG::2012-12-10 17:17:13,880::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-32::WARNING::2012-12-10 17:17:13,880::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5838074b-e545-49d3-a4c8-da1ecd091999_imageNS already registered >Thread-32::WARNING::2012-12-10 17:17:13,880::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5838074b-e545-49d3-a4c8-da1ecd091999_volumeNS already registered >Thread-32::INFO::2012-12-10 17:17:13,881::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '814587117568', 'disktotal': '1074600869888', 'mdafree': 0}} >Thread-32::DEBUG::2012-12-10 17:17:13,881::task::1172::TaskManager.Task::(prepare) Task=`ae62b0f8-3dbe-4be4-8a70-daeaa2827f73`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '814587117568', 'disktotal': '1074600869888', 'mdafree': 0}} >Thread-32::DEBUG::2012-12-10 17:17:13,882::task::588::TaskManager.Task::(_updateState) Task=`ae62b0f8-3dbe-4be4-8a70-daeaa2827f73`::moving from state preparing -> state finished >Thread-32::DEBUG::2012-12-10 17:17:13,882::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.5838074b-e545-49d3-a4c8-da1ecd091999': < ResourceRef 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999', isValid: 'True' obj: 'None'>} >Thread-32::DEBUG::2012-12-10 17:17:13,882::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-32::DEBUG::2012-12-10 17:17:13,882::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' >Thread-32::DEBUG::2012-12-10 17:17:13,882::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' (0 active users) >Thread-32::DEBUG::2012-12-10 17:17:13,882::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' is free, finding out if anyone is waiting for it. >Thread-32::DEBUG::2012-12-10 17:17:13,883::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999', Clearing records. >Thread-32::DEBUG::2012-12-10 17:17:13,883::task::978::TaskManager.Task::(_decref) Task=`ae62b0f8-3dbe-4be4-8a70-daeaa2827f73`::ref 0 aborting False >Thread-34::DEBUG::2012-12-10 17:17:15,462::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-34::DEBUG::2012-12-10 17:17:15,463::task::588::TaskManager.Task::(_updateState) Task=`a1a58e1a-fdb9-45b3-a331-b27207950d14`::moving from state init -> state preparing >Thread-34::INFO::2012-12-10 17:17:15,463::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-34::INFO::2012-12-10 17:17:15,463::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-34::DEBUG::2012-12-10 17:17:15,463::task::1172::TaskManager.Task::(prepare) Task=`a1a58e1a-fdb9-45b3-a331-b27207950d14`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-34::DEBUG::2012-12-10 17:17:15,463::task::588::TaskManager.Task::(_updateState) Task=`a1a58e1a-fdb9-45b3-a331-b27207950d14`::moving from state preparing -> state finished >Thread-34::DEBUG::2012-12-10 17:17:15,463::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-34::DEBUG::2012-12-10 17:17:15,463::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2012-12-10 17:17:15,464::task::978::TaskManager.Task::(_decref) Task=`a1a58e1a-fdb9-45b3-a331-b27207950d14`::ref 0 aborting False >Thread-35::DEBUG::2012-12-10 17:17:15,541::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-35::DEBUG::2012-12-10 17:17:15,541::task::588::TaskManager.Task::(_updateState) Task=`aacad216-968f-4069-8476-718437151626`::moving from state init -> state preparing >Thread-35::INFO::2012-12-10 17:17:15,541::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-35::DEBUG::2012-12-10 17:17:15,545::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257 /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257' (cwd None) >Thread-35::DEBUG::2012-12-10 17:17:15,562::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-35::DEBUG::2012-12-10 17:17:15,563::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-35::DEBUG::2012-12-10 17:17:15,563::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-35::DEBUG::2012-12-10 17:17:15,563::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-35::DEBUG::2012-12-10 17:17:15,563::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-35::DEBUG::2012-12-10 17:17:15,563::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-35::INFO::2012-12-10 17:17:15,564::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-35::DEBUG::2012-12-10 17:17:15,564::task::1172::TaskManager.Task::(prepare) Task=`aacad216-968f-4069-8476-718437151626`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-35::DEBUG::2012-12-10 17:17:15,564::task::588::TaskManager.Task::(_updateState) Task=`aacad216-968f-4069-8476-718437151626`::moving from state preparing -> state finished >Thread-35::DEBUG::2012-12-10 17:17:15,564::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-35::DEBUG::2012-12-10 17:17:15,564::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-35::DEBUG::2012-12-10 17:17:15,564::task::978::TaskManager.Task::(_decref) Task=`aacad216-968f-4069-8476-718437151626`::ref 0 aborting False >Thread-36::DEBUG::2012-12-10 17:17:15,639::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-36::DEBUG::2012-12-10 17:17:15,639::task::588::TaskManager.Task::(_updateState) Task=`5836fc43-970a-4666-b011-bb599cdb295c`::moving from state init -> state preparing >Thread-36::INFO::2012-12-10 17:17:15,640::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=1, storageType=1, remotePath='10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257', options=None) >Thread-36::DEBUG::2012-12-10 17:17:15,640::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-36::DEBUG::2012-12-10 17:17:15,640::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-36::DEBUG::2012-12-10 17:17:15,640::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-36::DEBUG::2012-12-10 17:17:15,640::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-36::DEBUG::2012-12-10 17:17:15,640::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-36::DEBUG::2012-12-10 17:17:15,651::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-36::DEBUG::2012-12-10 17:17:15,652::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-37::DEBUG::2012-12-10 17:17:15,911::task::588::TaskManager.Task::(_updateState) Task=`faef809a-9e59-4172-85b9-072894137ea1`::moving from state init -> state preparing >Thread-37::INFO::2012-12-10 17:17:15,911::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-37::INFO::2012-12-10 17:17:15,911::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-37::DEBUG::2012-12-10 17:17:15,911::task::1172::TaskManager.Task::(prepare) Task=`faef809a-9e59-4172-85b9-072894137ea1`::finished: {} >Thread-37::DEBUG::2012-12-10 17:17:15,911::task::588::TaskManager.Task::(_updateState) Task=`faef809a-9e59-4172-85b9-072894137ea1`::moving from state preparing -> state finished >Thread-37::DEBUG::2012-12-10 17:17:15,911::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-37::DEBUG::2012-12-10 17:17:15,912::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-37::DEBUG::2012-12-10 17:17:15,912::task::978::TaskManager.Task::(_decref) Task=`faef809a-9e59-4172-85b9-072894137ea1`::ref 0 aborting False >Thread-36::DEBUG::2012-12-10 17:17:17,486::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-36::DEBUG::2012-12-10 17:17:17,522::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-36::DEBUG::2012-12-10 17:17:17,522::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-36::DEBUG::2012-12-10 17:17:17,522::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-36::DEBUG::2012-12-10 17:17:17,522::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-36::DEBUG::2012-12-10 17:17:17,523::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-36::DEBUG::2012-12-10 17:17:17,523::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-36::DEBUG::2012-12-10 17:17:17,523::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-36::DEBUG::2012-12-10 17:17:17,523::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-36::DEBUG::2012-12-10 17:17:17,523::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-36::DEBUG::2012-12-10 17:17:17,524::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-36::DEBUG::2012-12-10 17:17:17,540::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-36::DEBUG::2012-12-10 17:17:17,541::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-36::INFO::2012-12-10 17:17:17,544::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-36::DEBUG::2012-12-10 17:17:17,544::task::1172::TaskManager.Task::(prepare) Task=`5836fc43-970a-4666-b011-bb599cdb295c`::finished: {'domlist': []} >Thread-36::DEBUG::2012-12-10 17:17:17,544::task::588::TaskManager.Task::(_updateState) Task=`5836fc43-970a-4666-b011-bb599cdb295c`::moving from state preparing -> state finished >Thread-36::DEBUG::2012-12-10 17:17:17,544::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-36::DEBUG::2012-12-10 17:17:17,544::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-36::DEBUG::2012-12-10 17:17:17,545::task::978::TaskManager.Task::(_decref) Task=`5836fc43-970a-4666-b011-bb599cdb295c`::ref 0 aborting False >Thread-41::DEBUG::2012-12-10 17:17:17,748::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-41::DEBUG::2012-12-10 17:17:17,748::task::588::TaskManager.Task::(_updateState) Task=`3493655a-2deb-4ec6-b58a-3e218b82d29e`::moving from state init -> state preparing >Thread-41::INFO::2012-12-10 17:17:17,748::logUtils::37::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='9f92a259-db12-4bfb-b35c-0e9e9c484fbf', domainName='nfsStorageTest_data_domain_1', typeSpecificArg='10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257', domClass=1, domVersion='3', options=None) >Thread-41::DEBUG::2012-12-10 17:17:17,748::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-41::DEBUG::2012-12-10 17:17:17,749::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9f92a259-db12-4bfb-b35c-0e9e9c484fbf' (cwd None) >Thread-41::DEBUG::2012-12-10 17:17:17,765::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "9f92a259-db12-4bfb-b35c-0e9e9c484fbf" not found\n'; <rc> = 5 >Thread-41::WARNING::2012-12-10 17:17:17,766::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "9f92a259-db12-4bfb-b35c-0e9e9c484fbf" not found'] >Thread-41::DEBUG::2012-12-10 17:17:17,766::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-41::INFO::2012-12-10 17:17:17,769::nfsSD::64::Storage.StorageDomain::(create) sdUUID=9f92a259-db12-4bfb-b35c-0e9e9c484fbf domainName=nfsStorageTest_data_domain_1 remotePath=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257 domClass=1 >Thread-41::DEBUG::2012-12-10 17:17:17,783::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2012-12-10 17:17:17,783::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-41::WARNING::2012-12-10 17:17:17,783::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-41::DEBUG::2012-12-10 17:17:17,784::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-41::DEBUG::2012-12-10 17:17:17,784::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-41::DEBUG::2012-12-10 17:17:17,784::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257', 'ROLE=Regular', 'SDUUID=9f92a259-db12-4bfb-b35c-0e9e9c484fbf', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=fb7111a1c165f877929b4e7f13a6582fa94e8423'] >Thread-41::DEBUG::2012-12-10 17:17:17,785::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-41::DEBUG::2012-12-10 17:17:17,786::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257/9f92a259-db12-4bfb-b35c-0e9e9c484fbf >Thread-41::DEBUG::2012-12-10 17:17:17,787::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2012-12-10 17:17:17,789::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257', 'ROLE=Regular', 'SDUUID=9f92a259-db12-4bfb-b35c-0e9e9c484fbf', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=fb7111a1c165f877929b4e7f13a6582fa94e8423'] >Thread-41::DEBUG::2012-12-10 17:17:17,790::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-41::DEBUG::2012-12-10 17:17:17,791::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '9f92a259-db12-4bfb-b35c-0e9e9c484fbf_imageNS' >Thread-41::DEBUG::2012-12-10 17:17:17,791::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '9f92a259-db12-4bfb-b35c-0e9e9c484fbf_volumeNS' >Thread-41::DEBUG::2012-12-10 17:17:17,812::sd::354::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-41::INFO::2012-12-10 17:17:17,812::logUtils::39::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-41::DEBUG::2012-12-10 17:17:17,812::task::1172::TaskManager.Task::(prepare) Task=`3493655a-2deb-4ec6-b58a-3e218b82d29e`::finished: None >Thread-41::DEBUG::2012-12-10 17:17:17,813::task::588::TaskManager.Task::(_updateState) Task=`3493655a-2deb-4ec6-b58a-3e218b82d29e`::moving from state preparing -> state finished >Thread-41::DEBUG::2012-12-10 17:17:17,813::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-41::DEBUG::2012-12-10 17:17:17,813::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-41::DEBUG::2012-12-10 17:17:17,813::task::978::TaskManager.Task::(_decref) Task=`3493655a-2deb-4ec6-b58a-3e218b82d29e`::ref 0 aborting False >Thread-44::DEBUG::2012-12-10 17:17:17,857::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-44::DEBUG::2012-12-10 17:17:17,858::task::588::TaskManager.Task::(_updateState) Task=`43d2805e-00c7-4992-8b23-3baf419e5f28`::moving from state init -> state preparing >Thread-44::INFO::2012-12-10 17:17:17,858::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='9f92a259-db12-4bfb-b35c-0e9e9c484fbf', options=None) >Thread-44::DEBUG::2012-12-10 17:17:17,858::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf`ReqID=`01b82574-0042-4a06-a327-693e0be36f54`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-44::DEBUG::2012-12-10 17:17:17,858::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf' for lock type 'shared' >Thread-44::DEBUG::2012-12-10 17:17:17,859::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf' is free. Now locking as 'shared' (1 active user) >Thread-44::DEBUG::2012-12-10 17:17:17,859::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf`ReqID=`01b82574-0042-4a06-a327-693e0be36f54`::Granted request >Thread-44::DEBUG::2012-12-10 17:17:17,859::task::817::TaskManager.Task::(resourceAcquired) Task=`43d2805e-00c7-4992-8b23-3baf419e5f28`::_resourcesAcquired: Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf (shared) >Thread-44::DEBUG::2012-12-10 17:17:17,859::task::978::TaskManager.Task::(_decref) Task=`43d2805e-00c7-4992-8b23-3baf419e5f28`::ref 1 aborting False >Thread-44::DEBUG::2012-12-10 17:17:17,859::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-44::DEBUG::2012-12-10 17:17:17,860::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 9f92a259-db12-4bfb-b35c-0e9e9c484fbf' (cwd None) >Thread-44::DEBUG::2012-12-10 17:17:17,876::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "9f92a259-db12-4bfb-b35c-0e9e9c484fbf" not found\n'; <rc> = 5 >Thread-44::WARNING::2012-12-10 17:17:17,876::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "9f92a259-db12-4bfb-b35c-0e9e9c484fbf" not found'] >Thread-44::DEBUG::2012-12-10 17:17:17,877::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-44::DEBUG::2012-12-10 17:17:17,883::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257/9f92a259-db12-4bfb-b35c-0e9e9c484fbf >Thread-44::DEBUG::2012-12-10 17:17:17,883::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-44::DEBUG::2012-12-10 17:17:17,885::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_55_179257', 'ROLE=Regular', 'SDUUID=9f92a259-db12-4bfb-b35c-0e9e9c484fbf', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=fb7111a1c165f877929b4e7f13a6582fa94e8423'] >Thread-44::DEBUG::2012-12-10 17:17:17,886::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-44::WARNING::2012-12-10 17:17:17,886::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 9f92a259-db12-4bfb-b35c-0e9e9c484fbf_imageNS already registered >Thread-44::WARNING::2012-12-10 17:17:17,886::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 9f92a259-db12-4bfb-b35c-0e9e9c484fbf_volumeNS already registered >Thread-44::INFO::2012-12-10 17:17:17,887::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '814579187712', 'disktotal': '1074600869888', 'mdafree': 0}} >Thread-44::DEBUG::2012-12-10 17:17:17,887::task::1172::TaskManager.Task::(prepare) Task=`43d2805e-00c7-4992-8b23-3baf419e5f28`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '814579187712', 'disktotal': '1074600869888', 'mdafree': 0}} >Thread-44::DEBUG::2012-12-10 17:17:17,887::task::588::TaskManager.Task::(_updateState) Task=`43d2805e-00c7-4992-8b23-3baf419e5f28`::moving from state preparing -> state finished >Thread-44::DEBUG::2012-12-10 17:17:17,887::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf': < ResourceRef 'Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf', isValid: 'True' obj: 'None'>} >Thread-44::DEBUG::2012-12-10 17:17:17,888::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-44::DEBUG::2012-12-10 17:17:17,888::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf' >Thread-44::DEBUG::2012-12-10 17:17:17,888::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf' (0 active users) >Thread-44::DEBUG::2012-12-10 17:17:17,888::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf' is free, finding out if anyone is waiting for it. >Thread-44::DEBUG::2012-12-10 17:17:17,888::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.9f92a259-db12-4bfb-b35c-0e9e9c484fbf', Clearing records. >Thread-44::DEBUG::2012-12-10 17:17:17,888::task::978::TaskManager.Task::(_decref) Task=`43d2805e-00c7-4992-8b23-3baf419e5f28`::ref 0 aborting False >Thread-48::DEBUG::2012-12-10 17:17:19,065::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-48::DEBUG::2012-12-10 17:17:19,065::task::588::TaskManager.Task::(_updateState) Task=`3fdcc5f7-5f7c-4327-b7de-4557ba522d0e`::moving from state init -> state preparing >Thread-48::INFO::2012-12-10 17:17:19,065::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-48::INFO::2012-12-10 17:17:19,065::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-48::DEBUG::2012-12-10 17:17:19,065::task::1172::TaskManager.Task::(prepare) Task=`3fdcc5f7-5f7c-4327-b7de-4557ba522d0e`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-48::DEBUG::2012-12-10 17:17:19,065::task::588::TaskManager.Task::(_updateState) Task=`3fdcc5f7-5f7c-4327-b7de-4557ba522d0e`::moving from state preparing -> state finished >Thread-48::DEBUG::2012-12-10 17:17:19,066::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-48::DEBUG::2012-12-10 17:17:19,066::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-48::DEBUG::2012-12-10 17:17:19,066::task::978::TaskManager.Task::(_decref) Task=`3fdcc5f7-5f7c-4327-b7de-4557ba522d0e`::ref 0 aborting False >Thread-49::DEBUG::2012-12-10 17:17:19,134::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-49::DEBUG::2012-12-10 17:17:19,135::task::588::TaskManager.Task::(_updateState) Task=`9a83cc1f-6ec0-4517-8214-3aec15602203`::moving from state init -> state preparing >Thread-49::INFO::2012-12-10 17:17:19,135::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-49::DEBUG::2012-12-10 17:17:19,139::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601 /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601' (cwd None) >Thread-49::DEBUG::2012-12-10 17:17:19,300::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49::DEBUG::2012-12-10 17:17:19,300::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49::DEBUG::2012-12-10 17:17:19,300::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49::DEBUG::2012-12-10 17:17:19,301::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49::DEBUG::2012-12-10 17:17:19,301::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49::DEBUG::2012-12-10 17:17:19,301::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49::INFO::2012-12-10 17:17:19,301::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-49::DEBUG::2012-12-10 17:17:19,301::task::1172::TaskManager.Task::(prepare) Task=`9a83cc1f-6ec0-4517-8214-3aec15602203`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-49::DEBUG::2012-12-10 17:17:19,302::task::588::TaskManager.Task::(_updateState) Task=`9a83cc1f-6ec0-4517-8214-3aec15602203`::moving from state preparing -> state finished >Thread-49::DEBUG::2012-12-10 17:17:19,302::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49::DEBUG::2012-12-10 17:17:19,302::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49::DEBUG::2012-12-10 17:17:19,302::task::978::TaskManager.Task::(_decref) Task=`9a83cc1f-6ec0-4517-8214-3aec15602203`::ref 0 aborting False >Thread-50::DEBUG::2012-12-10 17:17:19,414::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-50::DEBUG::2012-12-10 17:17:19,414::task::588::TaskManager.Task::(_updateState) Task=`3ed4bea8-00f8-4f3d-b691-3658ba704f25`::moving from state init -> state preparing >Thread-50::INFO::2012-12-10 17:17:19,414::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=2, storageType=1, remotePath='10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601', options=None) >Thread-50::DEBUG::2012-12-10 17:17:19,414::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-50::DEBUG::2012-12-10 17:17:19,414::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-50::DEBUG::2012-12-10 17:17:19,414::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-50::DEBUG::2012-12-10 17:17:19,415::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-50::DEBUG::2012-12-10 17:17:19,415::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-50::DEBUG::2012-12-10 17:17:19,426::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2012-12-10 17:17:19,426::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-50::DEBUG::2012-12-10 17:17:21,260::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-50::DEBUG::2012-12-10 17:17:21,295::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2012-12-10 17:17:21,295::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2012-12-10 17:17:21,295::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2012-12-10 17:17:21,295::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2012-12-10 17:17:21,296::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2012-12-10 17:17:21,296::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2012-12-10 17:17:21,296::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2012-12-10 17:17:21,296::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-50::DEBUG::2012-12-10 17:17:21,296::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2012-12-10 17:17:21,297::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-50::DEBUG::2012-12-10 17:17:21,312::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-50::DEBUG::2012-12-10 17:17:21,313::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-50::INFO::2012-12-10 17:17:21,316::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-50::DEBUG::2012-12-10 17:17:21,316::task::1172::TaskManager.Task::(prepare) Task=`3ed4bea8-00f8-4f3d-b691-3658ba704f25`::finished: {'domlist': []} >Thread-50::DEBUG::2012-12-10 17:17:21,316::task::588::TaskManager.Task::(_updateState) Task=`3ed4bea8-00f8-4f3d-b691-3658ba704f25`::moving from state preparing -> state finished >Thread-50::DEBUG::2012-12-10 17:17:21,316::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-50::DEBUG::2012-12-10 17:17:21,316::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-50::DEBUG::2012-12-10 17:17:21,316::task::978::TaskManager.Task::(_decref) Task=`3ed4bea8-00f8-4f3d-b691-3658ba704f25`::ref 0 aborting False >Thread-55::DEBUG::2012-12-10 17:17:21,483::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-55::DEBUG::2012-12-10 17:17:21,483::task::588::TaskManager.Task::(_updateState) Task=`23e8f14a-1fa7-4cb1-b15f-d79e1f0069f0`::moving from state init -> state preparing >Thread-55::INFO::2012-12-10 17:17:21,483::logUtils::37::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='f1bb75d5-6307-4498-bb65-de8dc17210b8', domainName='nfsStorageTest_ISO', typeSpecificArg='10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601', domClass=2, domVersion='0', options=None) >Thread-55::DEBUG::2012-12-10 17:17:21,484::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-55::DEBUG::2012-12-10 17:17:21,484::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free f1bb75d5-6307-4498-bb65-de8dc17210b8' (cwd None) >Thread-55::DEBUG::2012-12-10 17:17:21,500::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "f1bb75d5-6307-4498-bb65-de8dc17210b8" not found\n'; <rc> = 5 >Thread-55::WARNING::2012-12-10 17:17:21,501::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "f1bb75d5-6307-4498-bb65-de8dc17210b8" not found'] >Thread-55::DEBUG::2012-12-10 17:17:21,501::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-55::INFO::2012-12-10 17:17:21,504::nfsSD::64::Storage.StorageDomain::(create) sdUUID=f1bb75d5-6307-4498-bb65-de8dc17210b8 domainName=nfsStorageTest_ISO remotePath=10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601 domClass=2 >Thread-55::DEBUG::2012-12-10 17:17:21,607::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-55::DEBUG::2012-12-10 17:17:21,608::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-55::WARNING::2012-12-10 17:17:21,608::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-55::DEBUG::2012-12-10 17:17:21,609::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-55::DEBUG::2012-12-10 17:17:21,609::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-55::DEBUG::2012-12-10 17:17:21,609::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=nfsStorageTest_ISO', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601', 'ROLE=Regular', 'SDUUID=f1bb75d5-6307-4498-bb65-de8dc17210b8', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=31266abcad09aa1d7e168a808a992d5f176a638c'] >Thread-55::DEBUG::2012-12-10 17:17:21,611::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-55::DEBUG::2012-12-10 17:17:21,614::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601/f1bb75d5-6307-4498-bb65-de8dc17210b8 >Thread-55::DEBUG::2012-12-10 17:17:21,614::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-55::DEBUG::2012-12-10 17:17:21,617::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=nfsStorageTest_ISO', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601', 'ROLE=Regular', 'SDUUID=f1bb75d5-6307-4498-bb65-de8dc17210b8', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=31266abcad09aa1d7e168a808a992d5f176a638c'] >Thread-55::DEBUG::2012-12-10 17:17:21,618::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-55::DEBUG::2012-12-10 17:17:21,619::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'f1bb75d5-6307-4498-bb65-de8dc17210b8_imageNS' >Thread-55::DEBUG::2012-12-10 17:17:21,619::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'f1bb75d5-6307-4498-bb65-de8dc17210b8_volumeNS' >Thread-55::DEBUG::2012-12-10 17:17:21,619::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/libexec/vdsm/safelease release -f /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601/f1bb75d5-6307-4498-bb65-de8dc17210b8/dom_md/leases 0' (cwd /usr/libexec/vdsm) >Thread-55::DEBUG::2012-12-10 17:17:21,652::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-55::DEBUG::2012-12-10 17:17:21,653::sd::354::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-55::INFO::2012-12-10 17:17:21,653::logUtils::39::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-55::DEBUG::2012-12-10 17:17:21,653::task::1172::TaskManager.Task::(prepare) Task=`23e8f14a-1fa7-4cb1-b15f-d79e1f0069f0`::finished: None >Thread-55::DEBUG::2012-12-10 17:17:21,653::task::588::TaskManager.Task::(_updateState) Task=`23e8f14a-1fa7-4cb1-b15f-d79e1f0069f0`::moving from state preparing -> state finished >Thread-55::DEBUG::2012-12-10 17:17:21,653::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2012-12-10 17:17:21,653::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2012-12-10 17:17:21,654::task::978::TaskManager.Task::(_decref) Task=`23e8f14a-1fa7-4cb1-b15f-d79e1f0069f0`::ref 0 aborting False >Thread-59::DEBUG::2012-12-10 17:17:21,724::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-59::DEBUG::2012-12-10 17:17:21,725::task::588::TaskManager.Task::(_updateState) Task=`340b584b-dd8d-455c-b3d9-197f67309d76`::moving from state init -> state preparing >Thread-59::INFO::2012-12-10 17:17:21,725::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='f1bb75d5-6307-4498-bb65-de8dc17210b8', options=None) >Thread-59::DEBUG::2012-12-10 17:17:21,725::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8`ReqID=`01635b8c-7a2c-4e77-81c6-a8794b9dad8e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-59::DEBUG::2012-12-10 17:17:21,725::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8' for lock type 'shared' >Thread-59::DEBUG::2012-12-10 17:17:21,725::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8' is free. Now locking as 'shared' (1 active user) >Thread-59::DEBUG::2012-12-10 17:17:21,726::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8`ReqID=`01635b8c-7a2c-4e77-81c6-a8794b9dad8e`::Granted request >Thread-59::DEBUG::2012-12-10 17:17:21,726::task::817::TaskManager.Task::(resourceAcquired) Task=`340b584b-dd8d-455c-b3d9-197f67309d76`::_resourcesAcquired: Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8 (shared) >Thread-59::DEBUG::2012-12-10 17:17:21,726::task::978::TaskManager.Task::(_decref) Task=`340b584b-dd8d-455c-b3d9-197f67309d76`::ref 1 aborting False >Thread-59::DEBUG::2012-12-10 17:17:21,726::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-59::DEBUG::2012-12-10 17:17:21,727::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free f1bb75d5-6307-4498-bb65-de8dc17210b8' (cwd None) >Thread-59::DEBUG::2012-12-10 17:17:21,742::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "f1bb75d5-6307-4498-bb65-de8dc17210b8" not found\n'; <rc> = 5 >Thread-59::WARNING::2012-12-10 17:17:21,743::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "f1bb75d5-6307-4498-bb65-de8dc17210b8" not found'] >Thread-59::DEBUG::2012-12-10 17:17:21,743::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-59::DEBUG::2012-12-10 17:17:21,749::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601/f1bb75d5-6307-4498-bb65-de8dc17210b8 >Thread-59::DEBUG::2012-12-10 17:17:21,750::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-59::DEBUG::2012-12-10 17:17:21,752::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=nfsStorageTest_ISO', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121210171355962601', 'ROLE=Regular', 'SDUUID=f1bb75d5-6307-4498-bb65-de8dc17210b8', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=31266abcad09aa1d7e168a808a992d5f176a638c'] >Thread-59::DEBUG::2012-12-10 17:17:21,753::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-59::WARNING::2012-12-10 17:17:21,753::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f1bb75d5-6307-4498-bb65-de8dc17210b8_imageNS already registered >Thread-59::WARNING::2012-12-10 17:17:21,753::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f1bb75d5-6307-4498-bb65-de8dc17210b8_volumeNS already registered >Thread-59::INFO::2012-12-10 17:17:21,754::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '842267623424', 'disktotal': '842267623424', 'mdafree': 0}} >Thread-59::DEBUG::2012-12-10 17:17:21,754::task::1172::TaskManager.Task::(prepare) Task=`340b584b-dd8d-455c-b3d9-197f67309d76`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '842267623424', 'disktotal': '842267623424', 'mdafree': 0}} >Thread-59::DEBUG::2012-12-10 17:17:21,754::task::588::TaskManager.Task::(_updateState) Task=`340b584b-dd8d-455c-b3d9-197f67309d76`::moving from state preparing -> state finished >Thread-59::DEBUG::2012-12-10 17:17:21,754::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8': < ResourceRef 'Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8', isValid: 'True' obj: 'None'>} >Thread-59::DEBUG::2012-12-10 17:17:21,754::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-59::DEBUG::2012-12-10 17:17:21,755::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8' >Thread-59::DEBUG::2012-12-10 17:17:21,755::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8' (0 active users) >Thread-59::DEBUG::2012-12-10 17:17:21,755::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8' is free, finding out if anyone is waiting for it. >Thread-59::DEBUG::2012-12-10 17:17:21,755::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.f1bb75d5-6307-4498-bb65-de8dc17210b8', Clearing records. >Thread-59::DEBUG::2012-12-10 17:17:21,755::task::978::TaskManager.Task::(_decref) Task=`340b584b-dd8d-455c-b3d9-197f67309d76`::ref 0 aborting False >Thread-64::DEBUG::2012-12-10 17:17:22,923::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-64::DEBUG::2012-12-10 17:17:22,923::task::588::TaskManager.Task::(_updateState) Task=`634c4a0f-2e05-4160-8b19-ef17c438be8d`::moving from state init -> state preparing >Thread-64::INFO::2012-12-10 17:17:22,923::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4', 'port': ''}], options=None) >Thread-64::INFO::2012-12-10 17:17:22,923::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4'}]} >Thread-64::DEBUG::2012-12-10 17:17:22,923::task::1172::TaskManager.Task::(prepare) Task=`634c4a0f-2e05-4160-8b19-ef17c438be8d`::finished: {'statuslist': [{'status': 0, 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4'}]} >Thread-64::DEBUG::2012-12-10 17:17:22,924::task::588::TaskManager.Task::(_updateState) Task=`634c4a0f-2e05-4160-8b19-ef17c438be8d`::moving from state preparing -> state finished >Thread-64::DEBUG::2012-12-10 17:17:22,924::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-64::DEBUG::2012-12-10 17:17:22,924::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-64::DEBUG::2012-12-10 17:17:22,924::task::978::TaskManager.Task::(_decref) Task=`634c4a0f-2e05-4160-8b19-ef17c438be8d`::ref 0 aborting False >Thread-65::DEBUG::2012-12-10 17:17:22,987::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-65::DEBUG::2012-12-10 17:17:22,988::task::588::TaskManager.Task::(_updateState) Task=`0b37dcf6-9180-4f5c-bd30-998f5c4924c9`::moving from state init -> state preparing >Thread-65::INFO::2012-12-10 17:17:22,988::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4', 'port': ''}], options=None) >Thread-65::DEBUG::2012-12-10 17:17:22,991::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-65::DEBUG::2012-12-10 17:17:22,992::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-65::DEBUG::2012-12-10 17:17:22,992::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-65::DEBUG::2012-12-10 17:17:22,992::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-65::DEBUG::2012-12-10 17:17:22,992::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-65::DEBUG::2012-12-10 17:17:22,992::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-65::INFO::2012-12-10 17:17:22,992::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4'}]} >Thread-65::DEBUG::2012-12-10 17:17:22,993::task::1172::TaskManager.Task::(prepare) Task=`0b37dcf6-9180-4f5c-bd30-998f5c4924c9`::finished: {'statuslist': [{'status': 0, 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4'}]} >Thread-65::DEBUG::2012-12-10 17:17:22,993::task::588::TaskManager.Task::(_updateState) Task=`0b37dcf6-9180-4f5c-bd30-998f5c4924c9`::moving from state preparing -> state finished >Thread-65::DEBUG::2012-12-10 17:17:22,993::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-65::DEBUG::2012-12-10 17:17:22,993::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-65::DEBUG::2012-12-10 17:17:22,993::task::978::TaskManager.Task::(_decref) Task=`0b37dcf6-9180-4f5c-bd30-998f5c4924c9`::ref 0 aborting False >Thread-66::DEBUG::2012-12-10 17:17:23,024::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-66::DEBUG::2012-12-10 17:17:23,024::task::588::TaskManager.Task::(_updateState) Task=`4fa95ee8-c925-43c1-988f-227e37148097`::moving from state init -> state preparing >Thread-66::INFO::2012-12-10 17:17:23,024::logUtils::37::dispatcher::(wrapper) Run and protect: createStoragePool(poolType=None, spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', poolName='nfsStorageTest', masterDom='5838074b-e545-49d3-a4c8-da1ecd091999', domList=['5838074b-e545-49d3-a4c8-da1ecd091999'], masterVersion=1, lockPolicy=None, lockRenewalIntervalSec=5, leaseTimeSec=60, ioOpTimeoutSec=10, leaseRetries=3, options=None) >Thread-66::DEBUG::2012-12-10 17:17:23,025::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-66::DEBUG::2012-12-10 17:17:23,025::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2012-12-10 17:17:23,025::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-66::DEBUG::2012-12-10 17:17:23,025::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2012-12-10 17:17:23,025::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-66::DEBUG::2012-12-10 17:17:23,036::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2012-12-10 17:17:23,037::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-66::DEBUG::2012-12-10 17:17:24,871::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-66::DEBUG::2012-12-10 17:17:24,907::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2012-12-10 17:17:24,907::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:24,907::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:24,907::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:24,908::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:24,908::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:24,908::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:24,908::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-66::DEBUG::2012-12-10 17:17:24,908::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:24,909::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 5838074b-e545-49d3-a4c8-da1ecd091999' (cwd None) >Thread-66::DEBUG::2012-12-10 17:17:24,926::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "5838074b-e545-49d3-a4c8-da1ecd091999" not found\n'; <rc> = 5 >Thread-66::WARNING::2012-12-10 17:17:24,927::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "5838074b-e545-49d3-a4c8-da1ecd091999" not found'] >Thread-66::DEBUG::2012-12-10 17:17:24,927::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:24,933::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-66::DEBUG::2012-12-10 17:17:24,933::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-66::DEBUG::2012-12-10 17:17:24,934::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f4a6536b639f001552cb32b51970faa9bb89bca5'] >Thread-66::DEBUG::2012-12-10 17:17:24,936::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-66::WARNING::2012-12-10 17:17:24,936::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5838074b-e545-49d3-a4c8-da1ecd091999_imageNS already registered >Thread-66::WARNING::2012-12-10 17:17:24,936::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5838074b-e545-49d3-a4c8-da1ecd091999_volumeNS already registered >Thread-66::INFO::2012-12-10 17:17:24,936::fileSD::260::Storage.StorageDomain::(validate) sdUUID=5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-66::DEBUG::2012-12-10 17:17:24,938::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f4a6536b639f001552cb32b51970faa9bb89bca5'] >Thread-66::DEBUG::2012-12-10 17:17:24,938::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`35dc50ed-dc38-4674-b12d-04ff89067d3c`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-66::DEBUG::2012-12-10 17:17:24,938::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'exclusive' >Thread-66::DEBUG::2012-12-10 17:17:24,939::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'exclusive' (1 active user) >Thread-66::DEBUG::2012-12-10 17:17:24,939::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`35dc50ed-dc38-4674-b12d-04ff89067d3c`::Granted request >Thread-66::DEBUG::2012-12-10 17:17:24,939::task::817::TaskManager.Task::(resourceAcquired) Task=`4fa95ee8-c925-43c1-988f-227e37148097`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (exclusive) >Thread-66::DEBUG::2012-12-10 17:17:24,939::task::978::TaskManager.Task::(_decref) Task=`4fa95ee8-c925-43c1-988f-227e37148097`::ref 1 aborting False >Thread-66::DEBUG::2012-12-10 17:17:24,940::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.5838074b-e545-49d3-a4c8-da1ecd091999`ReqID=`98c457ea-f944-4947-b034-9006999b96e8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-66::DEBUG::2012-12-10 17:17:24,940::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' for lock type 'exclusive' >Thread-66::DEBUG::2012-12-10 17:17:24,940::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' is free. Now locking as 'exclusive' (1 active user) >Thread-66::DEBUG::2012-12-10 17:17:24,940::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.5838074b-e545-49d3-a4c8-da1ecd091999`ReqID=`98c457ea-f944-4947-b034-9006999b96e8`::Granted request >Thread-66::DEBUG::2012-12-10 17:17:24,940::task::817::TaskManager.Task::(resourceAcquired) Task=`4fa95ee8-c925-43c1-988f-227e37148097`::_resourcesAcquired: Storage.5838074b-e545-49d3-a4c8-da1ecd091999 (exclusive) >Thread-66::DEBUG::2012-12-10 17:17:24,940::task::978::TaskManager.Task::(_decref) Task=`4fa95ee8-c925-43c1-988f-227e37148097`::ref 1 aborting False >Thread-66::INFO::2012-12-10 17:17:24,941::sp::542::Storage.StoragePool::(create) spUUID=6011f774-ac87-4e98-a0c2-52860f84deef poolName=nfsStorageTest master_sd=5838074b-e545-49d3-a4c8-da1ecd091999 domList=['5838074b-e545-49d3-a4c8-da1ecd091999'] masterVersion=1 {'LEASETIMESEC': 60, 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3, 'LOCKRENEWALINTERVALSEC': 5} >Thread-66::INFO::2012-12-10 17:17:24,941::fileSD::260::Storage.StorageDomain::(validate) sdUUID=5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-66::DEBUG::2012-12-10 17:17:24,942::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f4a6536b639f001552cb32b51970faa9bb89bca5'] >Thread-66::DEBUG::2012-12-10 17:17:24,944::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f4a6536b639f001552cb32b51970faa9bb89bca5'] >Thread-66::DEBUG::2012-12-10 17:17:24,944::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-66::DEBUG::2012-12-10 17:17:24,945::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-66::DEBUG::2012-12-10 17:17:24,945::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1b432d0ddea27c1f0137591680debc8add7a8131'] >Thread-66::DEBUG::2012-12-10 17:17:24,947::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-66::INFO::2012-12-10 17:17:24,947::safelease::156::SANLock::(acquireHostId) Acquiring host id for domain 5838074b-e545-49d3-a4c8-da1ecd091999 (id: 250) >Thread-71::DEBUG::2012-12-10 17:17:26,464::task::588::TaskManager.Task::(_updateState) Task=`4a6da162-9e98-4467-9702-5eb8137a0e8a`::moving from state init -> state preparing >Thread-71::INFO::2012-12-10 17:17:26,464::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-71::INFO::2012-12-10 17:17:26,465::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-71::DEBUG::2012-12-10 17:17:26,465::task::1172::TaskManager.Task::(prepare) Task=`4a6da162-9e98-4467-9702-5eb8137a0e8a`::finished: {} >Thread-71::DEBUG::2012-12-10 17:17:26,465::task::588::TaskManager.Task::(_updateState) Task=`4a6da162-9e98-4467-9702-5eb8137a0e8a`::moving from state preparing -> state finished >Thread-71::DEBUG::2012-12-10 17:17:26,465::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-71::DEBUG::2012-12-10 17:17:26,465::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-71::DEBUG::2012-12-10 17:17:26,465::task::978::TaskManager.Task::(_decref) Task=`4a6da162-9e98-4467-9702-5eb8137a0e8a`::ref 0 aborting False >Thread-77::DEBUG::2012-12-10 17:17:36,829::task::588::TaskManager.Task::(_updateState) Task=`4945d75d-cbd1-4fbb-92ff-21d3cfba5df2`::moving from state init -> state preparing >Thread-77::INFO::2012-12-10 17:17:36,829::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-77::INFO::2012-12-10 17:17:36,829::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-77::DEBUG::2012-12-10 17:17:36,829::task::1172::TaskManager.Task::(prepare) Task=`4945d75d-cbd1-4fbb-92ff-21d3cfba5df2`::finished: {} >Thread-77::DEBUG::2012-12-10 17:17:36,830::task::588::TaskManager.Task::(_updateState) Task=`4945d75d-cbd1-4fbb-92ff-21d3cfba5df2`::moving from state preparing -> state finished >Thread-77::DEBUG::2012-12-10 17:17:36,830::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-77::DEBUG::2012-12-10 17:17:36,830::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-77::DEBUG::2012-12-10 17:17:36,830::task::978::TaskManager.Task::(_decref) Task=`4945d75d-cbd1-4fbb-92ff-21d3cfba5df2`::ref 0 aborting False >Thread-66::DEBUG::2012-12-10 17:17:45,951::safelease::166::SANLock::(acquireHostId) Host id for domain 5838074b-e545-49d3-a4c8-da1ecd091999 successfully acquired (id: 250) >Thread-66::INFO::2012-12-10 17:17:45,952::safelease::188::SANLock::(acquire) Acquiring cluster lock for domain 5838074b-e545-49d3-a4c8-da1ecd091999 (id: 250) >Thread-66::DEBUG::2012-12-10 17:17:45,991::safelease::213::SANLock::(acquire) Cluster lock for domain 5838074b-e545-49d3-a4c8-da1ecd091999 successfully acquired (id: 250) >Thread-66::INFO::2012-12-10 17:17:45,991::sp::721::Storage.StoragePool::(createMaster) setting master domain for spUUID 6011f774-ac87-4e98-a0c2-52860f84deef on sdUUID=5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-66::DEBUG::2012-12-10 17:17:45,991::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-66::DEBUG::2012-12-10 17:17:45,992::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-66::DEBUG::2012-12-10 17:17:45,996::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Regular', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1b432d0ddea27c1f0137591680debc8add7a8131'] >Thread-66::DEBUG::2012-12-10 17:17:45,997::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-66::DEBUG::2012-12-10 17:17:45,997::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-66::DEBUG::2012-12-10 17:17:45,997::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-66::DEBUG::2012-12-10 17:17:45,997::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-66::DEBUG::2012-12-10 17:17:45,997::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=6011f774-ac87-4e98-a0c2-52860f84deef', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Master', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1e172f25faeba2582ff7e380ed5b9173925aa4d1'] >Thread-66::DEBUG::2012-12-10 17:17:45,999::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-66::DEBUG::2012-12-10 17:17:45,999::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 5838074b-e545-49d3-a4c8-da1ecd091999 verified, version 1 >Thread-66::DEBUG::2012-12-10 17:17:45,999::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-66::DEBUG::2012-12-10 17:17:46,000::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-66::DEBUG::2012-12-10 17:17:46,002::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=6011f774-ac87-4e98-a0c2-52860f84deef', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Master', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1e172f25faeba2582ff7e380ed5b9173925aa4d1'] >Thread-66::DEBUG::2012-12-10 17:17:46,002::sp::1495::Storage.StoragePool::(updateMonitoringThreads) sp `6011f774-ac87-4e98-a0c2-52860f84deef` started monitoring domain `5838074b-e545-49d3-a4c8-da1ecd091999` >Thread-66::DEBUG::2012-12-10 17:17:46,002::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-66::DEBUG::2012-12-10 17:17:46,004::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-66::DEBUG::2012-12-10 17:17:46,004::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-66::DEBUG::2012-12-10 17:17:46,022::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-66::DEBUG::2012-12-10 17:17:46,022::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-66::INFO::2012-12-10 17:17:46,025::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999 to /rhev/data-center/6011f774-ac87-4e98-a0c2-52860f84deef/5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-66::INFO::2012-12-10 17:17:46,025::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999 to /rhev/data-center/6011f774-ac87-4e98-a0c2-52860f84deef/mastersd >Thread-66::INFO::2012-12-10 17:17:46,025::safelease::218::SANLock::(release) Releasing cluster lock for domain 5838074b-e545-49d3-a4c8-da1ecd091999 (id: 250) >Thread-66::DEBUG::2012-12-10 17:17:46,151::safelease::229::SANLock::(release) Cluster lock for domain 5838074b-e545-49d3-a4c8-da1ecd091999 successfully released (id: 250) >Thread-66::INFO::2012-12-10 17:17:46,152::safelease::171::SANLock::(releaseHostId) Releasing host id for domain 5838074b-e545-49d3-a4c8-da1ecd091999 (id: 250) >Thread-87::DEBUG::2012-12-10 17:17:47,171::task::588::TaskManager.Task::(_updateState) Task=`e5d85e38-826c-4ac6-847b-9d2e28f6e7c6`::moving from state init -> state preparing >Thread-87::INFO::2012-12-10 17:17:47,171::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-87::INFO::2012-12-10 17:17:47,171::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-87::DEBUG::2012-12-10 17:17:47,171::task::1172::TaskManager.Task::(prepare) Task=`e5d85e38-826c-4ac6-847b-9d2e28f6e7c6`::finished: {} >Thread-87::DEBUG::2012-12-10 17:17:47,171::task::588::TaskManager.Task::(_updateState) Task=`e5d85e38-826c-4ac6-847b-9d2e28f6e7c6`::moving from state preparing -> state finished >Thread-87::DEBUG::2012-12-10 17:17:47,172::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-87::DEBUG::2012-12-10 17:17:47,172::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-87::DEBUG::2012-12-10 17:17:47,172::task::978::TaskManager.Task::(_decref) Task=`e5d85e38-826c-4ac6-847b-9d2e28f6e7c6`::ref 0 aborting False >Thread-66::DEBUG::2012-12-10 17:17:49,153::safelease::180::SANLock::(releaseHostId) Host id for domain 5838074b-e545-49d3-a4c8-da1ecd091999 released successfully (id: None) >Thread-66::INFO::2012-12-10 17:17:49,153::logUtils::39::dispatcher::(wrapper) Run and protect: createStoragePool, Return response: True >Thread-66::DEBUG::2012-12-10 17:17:49,153::task::1172::TaskManager.Task::(prepare) Task=`4fa95ee8-c925-43c1-988f-227e37148097`::finished: True >Thread-66::DEBUG::2012-12-10 17:17:49,153::task::588::TaskManager.Task::(_updateState) Task=`4fa95ee8-c925-43c1-988f-227e37148097`::moving from state preparing -> state finished >Thread-66::DEBUG::2012-12-10 17:17:49,153::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>, 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999': < ResourceRef 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999', isValid: 'True' obj: 'None'>} >Thread-66::DEBUG::2012-12-10 17:17:49,154::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-66::DEBUG::2012-12-10 17:17:49,154::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-66::DEBUG::2012-12-10 17:17:49,154::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-66::DEBUG::2012-12-10 17:17:49,154::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-66::DEBUG::2012-12-10 17:17:49,154::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-66::DEBUG::2012-12-10 17:17:49,155::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' >Thread-66::DEBUG::2012-12-10 17:17:49,155::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' (0 active users) >Thread-66::DEBUG::2012-12-10 17:17:49,155::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' is free, finding out if anyone is waiting for it. >Thread-66::DEBUG::2012-12-10 17:17:49,155::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999', Clearing records. >Thread-66::DEBUG::2012-12-10 17:17:49,155::task::978::TaskManager.Task::(_decref) Task=`4fa95ee8-c925-43c1-988f-227e37148097`::ref 0 aborting False >Thread-90::DEBUG::2012-12-10 17:17:49,397::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-90::DEBUG::2012-12-10 17:17:49,397::task::588::TaskManager.Task::(_updateState) Task=`62e565e8-76ef-4cef-b47b-54a67d2f3e7c`::moving from state init -> state preparing >Thread-90::INFO::2012-12-10 17:17:49,397::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4', 'port': ''}], options=None) >Thread-90::INFO::2012-12-10 17:17:49,397::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4'}]} >Thread-90::DEBUG::2012-12-10 17:17:49,397::task::1172::TaskManager.Task::(prepare) Task=`62e565e8-76ef-4cef-b47b-54a67d2f3e7c`::finished: {'statuslist': [{'status': 0, 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4'}]} >Thread-90::DEBUG::2012-12-10 17:17:49,398::task::588::TaskManager.Task::(_updateState) Task=`62e565e8-76ef-4cef-b47b-54a67d2f3e7c`::moving from state preparing -> state finished >Thread-90::DEBUG::2012-12-10 17:17:49,398::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2012-12-10 17:17:49,398::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2012-12-10 17:17:49,398::task::978::TaskManager.Task::(_decref) Task=`62e565e8-76ef-4cef-b47b-54a67d2f3e7c`::ref 0 aborting False >Thread-91::DEBUG::2012-12-10 17:17:49,452::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-91::DEBUG::2012-12-10 17:17:49,452::task::588::TaskManager.Task::(_updateState) Task=`c2ef1368-5e8b-45c9-acdd-657a473cbf47`::moving from state init -> state preparing >Thread-91::INFO::2012-12-10 17:17:49,453::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4', 'port': ''}], options=None) >Thread-91::DEBUG::2012-12-10 17:17:49,456::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-10 17:17:49,456::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::DEBUG::2012-12-10 17:17:49,457::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-10 17:17:49,457::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::DEBUG::2012-12-10 17:17:49,457::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-10 17:17:49,457::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::INFO::2012-12-10 17:17:49,457::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4'}]} >Thread-91::DEBUG::2012-12-10 17:17:49,457::task::1172::TaskManager.Task::(prepare) Task=`c2ef1368-5e8b-45c9-acdd-657a473cbf47`::finished: {'statuslist': [{'status': 0, 'id': '3dd3737f-0ee2-4307-93a2-032442366cf4'}]} >Thread-91::DEBUG::2012-12-10 17:17:49,458::task::588::TaskManager.Task::(_updateState) Task=`c2ef1368-5e8b-45c9-acdd-657a473cbf47`::moving from state preparing -> state finished >Thread-91::DEBUG::2012-12-10 17:17:49,458::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-91::DEBUG::2012-12-10 17:17:49,458::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-91::DEBUG::2012-12-10 17:17:49,458::task::978::TaskManager.Task::(_decref) Task=`c2ef1368-5e8b-45c9-acdd-657a473cbf47`::ref 0 aborting False >Thread-92::DEBUG::2012-12-10 17:17:49,573::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-92::DEBUG::2012-12-10 17:17:49,574::task::588::TaskManager.Task::(_updateState) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::moving from state init -> state preparing >Thread-92::INFO::2012-12-10 17:17:49,574::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-92::ERROR::2012-12-10 17:17:49,574::task::853::TaskManager.Task::(_setError) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 861, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 38, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 528, in getSpmStatus > pool = self.getPool(spUUID) > File "/usr/share/vdsm/storage/hsm.py", line 265, in getPool > raise se.StoragePoolUnknown(spUUID) >StoragePoolUnknown: Unknown pool id, pool not connected: ('6011f774-ac87-4e98-a0c2-52860f84deef',) >Thread-92::DEBUG::2012-12-10 17:17:49,575::task::872::TaskManager.Task::(_run) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::Task._run: 0e1426de-11ea-4d86-a693-ecdde46cd75d ('6011f774-ac87-4e98-a0c2-52860f84deef',) {} failed - stopping task >Thread-92::DEBUG::2012-12-10 17:17:49,575::task::1199::TaskManager.Task::(stop) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::stopping in state preparing (force False) >Thread-92::DEBUG::2012-12-10 17:17:49,576::task::978::TaskManager.Task::(_decref) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::ref 1 aborting True >Thread-92::INFO::2012-12-10 17:17:49,576::task::1157::TaskManager.Task::(prepare) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-92::DEBUG::2012-12-10 17:17:49,576::task::1162::TaskManager.Task::(prepare) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::Prepare: aborted: Unknown pool id, pool not connected >Thread-92::DEBUG::2012-12-10 17:17:49,576::task::978::TaskManager.Task::(_decref) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::ref 0 aborting True >Thread-92::DEBUG::2012-12-10 17:17:49,576::task::913::TaskManager.Task::(_doAbort) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::Task._doAbort: force False >Thread-92::DEBUG::2012-12-10 17:17:49,576::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2012-12-10 17:17:49,576::task::588::TaskManager.Task::(_updateState) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::moving from state preparing -> state aborting >Thread-92::DEBUG::2012-12-10 17:17:49,577::task::537::TaskManager.Task::(__state_aborting) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::_aborting: recover policy none >Thread-92::DEBUG::2012-12-10 17:17:49,577::task::588::TaskManager.Task::(_updateState) Task=`0e1426de-11ea-4d86-a693-ecdde46cd75d`::moving from state aborting -> state failed >Thread-92::DEBUG::2012-12-10 17:17:49,577::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2012-12-10 17:17:49,577::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::ERROR::2012-12-10 17:17:49,577::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('6011f774-ac87-4e98-a0c2-52860f84deef',)", 'code': 309}} >Thread-93::DEBUG::2012-12-10 17:17:49,617::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-93::DEBUG::2012-12-10 17:17:49,618::task::588::TaskManager.Task::(_updateState) Task=`c885bd6c-46fb-45ee-9ffa-27b08fee775d`::moving from state init -> state preparing >Thread-93::INFO::2012-12-10 17:17:49,618::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', hostID=1, scsiKey='6011f774-ac87-4e98-a0c2-52860f84deef', msdUUID='5838074b-e545-49d3-a4c8-da1ecd091999', masterVersion=1, options=None) >Thread-93::DEBUG::2012-12-10 17:17:49,618::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`ad6d1430-b4fa-4ce2-8fe4-1156720e1d9a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-93::DEBUG::2012-12-10 17:17:49,618::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'exclusive' >Thread-93::DEBUG::2012-12-10 17:17:49,619::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'exclusive' (1 active user) >Thread-93::DEBUG::2012-12-10 17:17:49,619::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`ad6d1430-b4fa-4ce2-8fe4-1156720e1d9a`::Granted request >Thread-93::INFO::2012-12-10 17:17:49,619::sp::625::Storage.StoragePool::(connect) Connect host #1 to the storage pool 6011f774-ac87-4e98-a0c2-52860f84deef with master domain: 5838074b-e545-49d3-a4c8-da1ecd091999 (ver = 1) >Thread-93::DEBUG::2012-12-10 17:17:49,619::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:49,620::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:49,620::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:49,620::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:49,620::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:49,620::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:49,620::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-93::DEBUG::2012-12-10 17:17:49,621::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-93::DEBUG::2012-12-10 17:17:49,621::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-93::DEBUG::2012-12-10 17:17:49,621::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-93::DEBUG::2012-12-10 17:17:49,621::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-93::DEBUG::2012-12-10 17:17:49,632::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-93::DEBUG::2012-12-10 17:17:49,633::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-93::DEBUG::2012-12-10 17:17:51,467::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-93::DEBUG::2012-12-10 17:17:51,504::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-93::DEBUG::2012-12-10 17:17:51,505::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,505::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,505::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,505::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,505::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,505::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,506::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-93::DEBUG::2012-12-10 17:17:51,506::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,506::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 5838074b-e545-49d3-a4c8-da1ecd091999' (cwd None) >Thread-93::DEBUG::2012-12-10 17:17:51,523::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "5838074b-e545-49d3-a4c8-da1ecd091999" not found\n'; <rc> = 5 >Thread-93::WARNING::2012-12-10 17:17:51,523::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "5838074b-e545-49d3-a4c8-da1ecd091999" not found'] >Thread-93::DEBUG::2012-12-10 17:17:51,524::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,529::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-93::DEBUG::2012-12-10 17:17:51,529::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-93::DEBUG::2012-12-10 17:17:51,531::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=6011f774-ac87-4e98-a0c2-52860f84deef', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Master', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1e172f25faeba2582ff7e380ed5b9173925aa4d1'] >Thread-93::DEBUG::2012-12-10 17:17:51,532::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-93::WARNING::2012-12-10 17:17:51,532::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5838074b-e545-49d3-a4c8-da1ecd091999_imageNS already registered >Thread-93::WARNING::2012-12-10 17:17:51,532::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5838074b-e545-49d3-a4c8-da1ecd091999_volumeNS already registered >Thread-93::DEBUG::2012-12-10 17:17:51,533::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 5838074b-e545-49d3-a4c8-da1ecd091999 verified, version 1 >Thread-93::DEBUG::2012-12-10 17:17:51,533::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-93::DEBUG::2012-12-10 17:17:51,533::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-93::DEBUG::2012-12-10 17:17:51,534::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=6011f774-ac87-4e98-a0c2-52860f84deef', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Master', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1e172f25faeba2582ff7e380ed5b9173925aa4d1'] >Thread-93::DEBUG::2012-12-10 17:17:51,535::sp::1495::Storage.StoragePool::(updateMonitoringThreads) sp `6011f774-ac87-4e98-a0c2-52860f84deef` started monitoring domain `5838074b-e545-49d3-a4c8-da1ecd091999` >Thread-93::DEBUG::2012-12-10 17:17:51,536::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-93::DEBUG::2012-12-10 17:17:51,538::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-93::DEBUG::2012-12-10 17:17:51,538::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-93::DEBUG::2012-12-10 17:17:51,555::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-93::DEBUG::2012-12-10 17:17:51,555::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-93::INFO::2012-12-10 17:17:51,558::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999 to /rhev/data-center/6011f774-ac87-4e98-a0c2-52860f84deef/5838074b-e545-49d3-a4c8-da1ecd091999 >Thread-93::INFO::2012-12-10 17:17:51,558::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999 to /rhev/data-center/6011f774-ac87-4e98-a0c2-52860f84deef/mastersd >Thread-93::INFO::2012-12-10 17:17:51,559::safelease::156::SANLock::(acquireHostId) Acquiring host id for domain 5838074b-e545-49d3-a4c8-da1ecd091999 (id: 1) >Thread-104::DEBUG::2012-12-10 17:17:57,596::task::588::TaskManager.Task::(_updateState) Task=`5b8f82d6-4641-44fc-9f4c-dea1aebedb15`::moving from state init -> state preparing >Thread-104::INFO::2012-12-10 17:17:57,596::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-104::INFO::2012-12-10 17:17:57,596::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-104::DEBUG::2012-12-10 17:17:57,597::task::1172::TaskManager.Task::(prepare) Task=`5b8f82d6-4641-44fc-9f4c-dea1aebedb15`::finished: {} >Thread-104::DEBUG::2012-12-10 17:17:57,597::task::588::TaskManager.Task::(_updateState) Task=`5b8f82d6-4641-44fc-9f4c-dea1aebedb15`::moving from state preparing -> state finished >Thread-104::DEBUG::2012-12-10 17:17:57,597::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-104::DEBUG::2012-12-10 17:17:57,597::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-104::DEBUG::2012-12-10 17:17:57,597::task::978::TaskManager.Task::(_decref) Task=`5b8f82d6-4641-44fc-9f4c-dea1aebedb15`::ref 0 aborting False >Thread-110::DEBUG::2012-12-10 17:18:07,951::task::588::TaskManager.Task::(_updateState) Task=`23d445ab-d856-47c9-b35c-d3255f452b92`::moving from state init -> state preparing >Thread-110::INFO::2012-12-10 17:18:07,951::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-110::INFO::2012-12-10 17:18:07,951::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-110::DEBUG::2012-12-10 17:18:07,951::task::1172::TaskManager.Task::(prepare) Task=`23d445ab-d856-47c9-b35c-d3255f452b92`::finished: {} >Thread-110::DEBUG::2012-12-10 17:18:07,951::task::588::TaskManager.Task::(_updateState) Task=`23d445ab-d856-47c9-b35c-d3255f452b92`::moving from state preparing -> state finished >Thread-110::DEBUG::2012-12-10 17:18:07,951::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2012-12-10 17:18:07,952::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2012-12-10 17:18:07,952::task::978::TaskManager.Task::(_decref) Task=`23d445ab-d856-47c9-b35c-d3255f452b92`::ref 0 aborting False >Thread-93::DEBUG::2012-12-10 17:18:12,563::safelease::166::SANLock::(acquireHostId) Host id for domain 5838074b-e545-49d3-a4c8-da1ecd091999 successfully acquired (id: 1) >Thread-93::DEBUG::2012-12-10 17:18:12,563::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-93::DEBUG::2012-12-10 17:18:12,564::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-93::DEBUG::2012-12-10 17:18:12,564::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-93::DEBUG::2012-12-10 17:18:12,564::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-93::INFO::2012-12-10 17:18:12,564::logUtils::39::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-93::DEBUG::2012-12-10 17:18:12,564::task::1172::TaskManager.Task::(prepare) Task=`c885bd6c-46fb-45ee-9ffa-27b08fee775d`::finished: True >Thread-93::DEBUG::2012-12-10 17:18:12,564::task::588::TaskManager.Task::(_updateState) Task=`c885bd6c-46fb-45ee-9ffa-27b08fee775d`::moving from state preparing -> state finished >Thread-93::DEBUG::2012-12-10 17:18:12,564::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2012-12-10 17:18:12,564::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2012-12-10 17:18:12,565::task::978::TaskManager.Task::(_decref) Task=`c885bd6c-46fb-45ee-9ffa-27b08fee775d`::ref 0 aborting False >Thread-112::DEBUG::2012-12-10 17:18:12,618::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-112::DEBUG::2012-12-10 17:18:12,618::task::588::TaskManager.Task::(_updateState) Task=`7d145d9e-2733-4e70-a910-9b3b4f84c1fe`::moving from state init -> state preparing >Thread-112::INFO::2012-12-10 17:18:12,618::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-112::INFO::2012-12-10 17:18:12,618::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-112::DEBUG::2012-12-10 17:18:12,618::task::1172::TaskManager.Task::(prepare) Task=`7d145d9e-2733-4e70-a910-9b3b4f84c1fe`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-112::DEBUG::2012-12-10 17:18:12,619::task::588::TaskManager.Task::(_updateState) Task=`7d145d9e-2733-4e70-a910-9b3b4f84c1fe`::moving from state preparing -> state finished >Thread-112::DEBUG::2012-12-10 17:18:12,619::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-112::DEBUG::2012-12-10 17:18:12,619::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-112::DEBUG::2012-12-10 17:18:12,619::task::978::TaskManager.Task::(_decref) Task=`7d145d9e-2733-4e70-a910-9b3b4f84c1fe`::ref 0 aborting False >Thread-113::DEBUG::2012-12-10 17:18:12,750::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-113::DEBUG::2012-12-10 17:18:12,750::task::588::TaskManager.Task::(_updateState) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::moving from state init -> state preparing >Thread-113::INFO::2012-12-10 17:18:12,750::logUtils::37::dispatcher::(wrapper) Run and protect: spmStart(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', prevID=-1, prevLVER='-1', recoveryMode=None, scsiFencing='false', maxHostID=250, domVersion='3', options=None) >Thread-113::DEBUG::2012-12-10 17:18:12,751::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`8bc70575-9e7b-498e-9ac2-52983c904bc6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-113::DEBUG::2012-12-10 17:18:12,751::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'exclusive' >Thread-113::DEBUG::2012-12-10 17:18:12,751::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'exclusive' (1 active user) >Thread-113::DEBUG::2012-12-10 17:18:12,751::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`8bc70575-9e7b-498e-9ac2-52983c904bc6`::Granted request >Thread-113::DEBUG::2012-12-10 17:18:12,751::task::817::TaskManager.Task::(resourceAcquired) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (exclusive) >Thread-113::DEBUG::2012-12-10 17:18:12,752::task::978::TaskManager.Task::(_decref) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::ref 1 aborting False >Thread-113::INFO::2012-12-10 17:18:12,752::logUtils::39::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >Thread-113::DEBUG::2012-12-10 17:18:12,752::task::1167::TaskManager.Task::(prepare) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::Prepare: 1 jobs exist, move to acquiring >Thread-113::DEBUG::2012-12-10 17:18:12,752::task::588::TaskManager.Task::(_updateState) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::moving from state preparing -> state acquiring >Thread-113::DEBUG::2012-12-10 17:18:12,752::task::588::TaskManager.Task::(_updateState) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::moving from state acquiring -> state queued >Thread-113::DEBUG::2012-12-10 17:18:12,752::taskManager::48::TaskManager::(_queueTask) queuing task: 4045044c-0ebd-417b-8fab-40c1acc5b08e >Thread-113::DEBUG::2012-12-10 17:18:12,752::taskManager::54::TaskManager::(_queueTask) task queued: 4045044c-0ebd-417b-8fab-40c1acc5b08e >Thread-1::DEBUG::2012-12-10 17:18:12,753::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-113::DEBUG::2012-12-10 17:18:12,753::task::1169::TaskManager.Task::(prepare) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::returning >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,753::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: 4045044c-0ebd-417b-8fab-40c1acc5b08e running: <bound method Task.commit of <storage.task.Task instance at 0x7f81282778c0>> with: None >Thread-113::DEBUG::2012-12-10 17:18:12,753::task::978::TaskManager.Task::(_decref) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::ref 0 aborting False >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,753::task::1181::TaskManager.Task::(commit) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::committing task: 4045044c-0ebd-417b-8fab-40c1acc5b08e >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,754::task::588::TaskManager.Task::(_updateState) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::moving from state queued -> state running >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,754::task::892::TaskManager.Task::(_runJobs) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f812827ac90>> (args: (-1, '-1', 'false', 250, 3) kwargs: {}) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,754::task::317::TaskManager.Task::(run) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f812827ac90>> (args: (-1, '-1', 'false', 250, 3) kwargs: {}) callback None >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,755::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,755::misc::1055::SamplingMethod::(__call__) Got in to sampling method >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,756::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=6011f774-ac87-4e98-a0c2-52860f84deef', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Master', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1e172f25faeba2582ff7e380ed5b9173925aa4d1'] >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,757::misc::1063::SamplingMethod::(__call__) Returning last result >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,758::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=6011f774-ac87-4e98-a0c2-52860f84deef', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Master', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1e172f25faeba2582ff7e380ed5b9173925aa4d1'] >4045044c-0ebd-417b-8fab-40c1acc5b08e::INFO::2012-12-10 17:18:12,759::safelease::188::SANLock::(acquire) Acquiring cluster lock for domain 5838074b-e545-49d3-a4c8-da1ecd091999 (id: 1) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,789::safelease::213::SANLock::(acquire) Cluster lock for domain 5838074b-e545-49d3-a4c8-da1ecd091999 successfully acquired (id: 1) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,790::sp::260::Storage.StoragePool::(startSpm) spm lock acquired successfully >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,791::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=6011f774-ac87-4e98-a0c2-52860f84deef', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Master', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=1e172f25faeba2582ff7e380ed5b9173925aa4d1'] >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,791::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,791::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,792::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=6011f774-ac87-4e98-a0c2-52860f84deef', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_10_17_13_53_821386', 'ROLE=Master', 'SDUUID=5838074b-e545-49d3-a4c8-da1ecd091999', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=b2189ac98c7d80803875c8b7a6ecde0a9087cc8d'] >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,793::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,793::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`bc7b9b82-154d-4ceb-8e49-d96103f5d9fc`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,793::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'exclusive' >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,794::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'exclusive' (1 active user) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,794::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`bc7b9b82-154d-4ceb-8e49-d96103f5d9fc`::Granted request >4045044c-0ebd-417b-8fab-40c1acc5b08e::INFO::2012-12-10 17:18:12,794::sp::407::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `5838074b-e545-49d3-a4c8-da1ecd091999` >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,794::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.5838074b-e545-49d3-a4c8-da1ecd091999`ReqID=`9b051696-6334-4923-bf60-7830768252b5`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,794::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' for lock type 'exclusive' >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,795::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' is free. Now locking as 'exclusive' (1 active user) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,795::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.5838074b-e545-49d3-a4c8-da1ecd091999`ReqID=`9b051696-6334-4923-bf60-7830768252b5`::Granted request >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,795::sd::385::Storage.StorageDomain::(upgrade) Trying to upgrade domain `5838074b-e545-49d3-a4c8-da1ecd091999` from version 3 to version 3 >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,795::sd::391::Storage.StorageDomain::(upgrade) No need to upgrade domain `5838074b-e545-49d3-a4c8-da1ecd091999`, leaving unchanged >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,795::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,795::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' (0 active users) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,796::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' is free, finding out if anyone is waiting for it. >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,796::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999', Clearing records. >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,796::sp::411::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,796::sp::418::Storage.StoragePool::(_upgradePool) Registering with state change event >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,796::sp::420::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,796::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_6011f774-ac87-4e98-a0c2-52860f84deef' >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,797::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,797::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,797::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >4045044c-0ebd-417b-8fab-40c1acc5b08e::WARNING::2012-12-10 17:18:12,798::sd::564::Storage.StorageDomain::(createMasterTree) vms dir not found, creating (/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999/master/vms) >4045044c-0ebd-417b-8fab-40c1acc5b08e::WARNING::2012-12-10 17:18:12,799::sd::569::Storage.StorageDomain::(createMasterTree) tasks dir not found, creating (/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386/5838074b-e545-49d3-a4c8-da1ecd091999/master/tasks) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,800::sp::309::Storage.StoragePool::(startSpm) ended. >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,800::task::588::TaskManager.Task::(_updateState) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::moving from state running -> state finished >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,800::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>} >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,800::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,801::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,801::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,801::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,801::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,801::task::900::TaskManager.Task::(_runJobs) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::Task.run: exit - success: result >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,801::task::978::TaskManager.Task::(_decref) Task=`4045044c-0ebd-417b-8fab-40c1acc5b08e`::ref 0 aborting False >4045044c-0ebd-417b-8fab-40c1acc5b08e::DEBUG::2012-12-10 17:18:12,802::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-114::DEBUG::2012-12-10 17:18:13,818::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-114::DEBUG::2012-12-10 17:18:13,819::task::588::TaskManager.Task::(_updateState) Task=`1deba9fb-0bc1-40c3-8680-69fe62d3f7fa`::moving from state init -> state preparing >Thread-114::INFO::2012-12-10 17:18:13,819::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='4045044c-0ebd-417b-8fab-40c1acc5b08e', spUUID=None, options=None) >Thread-114::DEBUG::2012-12-10 17:18:13,819::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 4045044c-0ebd-417b-8fab-40c1acc5b08e >Thread-114::DEBUG::2012-12-10 17:18:13,819::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '4045044c-0ebd-417b-8fab-40c1acc5b08e'} >Thread-114::INFO::2012-12-10 17:18:13,819::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '4045044c-0ebd-417b-8fab-40c1acc5b08e'}} >Thread-114::DEBUG::2012-12-10 17:18:13,819::task::1172::TaskManager.Task::(prepare) Task=`1deba9fb-0bc1-40c3-8680-69fe62d3f7fa`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '4045044c-0ebd-417b-8fab-40c1acc5b08e'}} >Thread-114::DEBUG::2012-12-10 17:18:13,820::task::588::TaskManager.Task::(_updateState) Task=`1deba9fb-0bc1-40c3-8680-69fe62d3f7fa`::moving from state preparing -> state finished >Thread-114::DEBUG::2012-12-10 17:18:13,820::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-114::DEBUG::2012-12-10 17:18:13,820::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-114::DEBUG::2012-12-10 17:18:13,820::task::978::TaskManager.Task::(_decref) Task=`1deba9fb-0bc1-40c3-8680-69fe62d3f7fa`::ref 0 aborting False >Thread-115::DEBUG::2012-12-10 17:18:13,882::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-115::DEBUG::2012-12-10 17:18:13,883::task::588::TaskManager.Task::(_updateState) Task=`aff9603c-44c1-43ae-a71e-938667bfae9c`::moving from state init -> state preparing >Thread-115::INFO::2012-12-10 17:18:13,883::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-115::INFO::2012-12-10 17:18:13,883::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-115::DEBUG::2012-12-10 17:18:13,883::task::1172::TaskManager.Task::(prepare) Task=`aff9603c-44c1-43ae-a71e-938667bfae9c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-115::DEBUG::2012-12-10 17:18:13,883::task::588::TaskManager.Task::(_updateState) Task=`aff9603c-44c1-43ae-a71e-938667bfae9c`::moving from state preparing -> state finished >Thread-115::DEBUG::2012-12-10 17:18:13,883::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2012-12-10 17:18:13,883::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2012-12-10 17:18:13,884::task::978::TaskManager.Task::(_decref) Task=`aff9603c-44c1-43ae-a71e-938667bfae9c`::ref 0 aborting False >Thread-116::DEBUG::2012-12-10 17:18:13,927::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-116::DEBUG::2012-12-10 17:18:13,928::task::588::TaskManager.Task::(_updateState) Task=`07cda8b4-782f-489f-9f45-3f004ba5ff42`::moving from state init -> state preparing >Thread-116::INFO::2012-12-10 17:18:13,928::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='4045044c-0ebd-417b-8fab-40c1acc5b08e', spUUID=None, options=None) >Thread-116::DEBUG::2012-12-10 17:18:13,928::taskManager::161::TaskManager::(clearTask) Entry. taskID: 4045044c-0ebd-417b-8fab-40c1acc5b08e >Thread-116::DEBUG::2012-12-10 17:18:13,928::taskManager::166::TaskManager::(clearTask) Return. >Thread-116::INFO::2012-12-10 17:18:13,928::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-116::DEBUG::2012-12-10 17:18:13,928::task::1172::TaskManager.Task::(prepare) Task=`07cda8b4-782f-489f-9f45-3f004ba5ff42`::finished: None >Thread-116::DEBUG::2012-12-10 17:18:13,928::task::588::TaskManager.Task::(_updateState) Task=`07cda8b4-782f-489f-9f45-3f004ba5ff42`::moving from state preparing -> state finished >Thread-116::DEBUG::2012-12-10 17:18:13,928::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-116::DEBUG::2012-12-10 17:18:13,929::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-116::DEBUG::2012-12-10 17:18:13,929::task::978::TaskManager.Task::(_decref) Task=`07cda8b4-782f-489f-9f45-3f004ba5ff42`::ref 0 aborting False >Thread-117::DEBUG::2012-12-10 17:18:13,988::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-117::DEBUG::2012-12-10 17:18:13,989::task::588::TaskManager.Task::(_updateState) Task=`350f3004-f0ba-4399-af6a-95b7daa28ab7`::moving from state init -> state preparing >Thread-117::INFO::2012-12-10 17:18:13,989::logUtils::37::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='5838074b-e545-49d3-a4c8-da1ecd091999', spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-117::DEBUG::2012-12-10 17:18:13,989::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`12278048-af2c-4454-86e6-0e9bb2c210a1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-117::DEBUG::2012-12-10 17:18:13,989::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'exclusive' >Thread-117::DEBUG::2012-12-10 17:18:13,990::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'exclusive' (1 active user) >Thread-117::DEBUG::2012-12-10 17:18:13,990::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`12278048-af2c-4454-86e6-0e9bb2c210a1`::Granted request >Thread-117::DEBUG::2012-12-10 17:18:13,990::task::817::TaskManager.Task::(resourceAcquired) Task=`350f3004-f0ba-4399-af6a-95b7daa28ab7`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (exclusive) >Thread-117::DEBUG::2012-12-10 17:18:13,990::task::978::TaskManager.Task::(_decref) Task=`350f3004-f0ba-4399-af6a-95b7daa28ab7`::ref 1 aborting False >Thread-117::DEBUG::2012-12-10 17:18:13,990::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.5838074b-e545-49d3-a4c8-da1ecd091999`ReqID=`6de16787-cb30-4784-b8fb-3fca915475c9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-117::DEBUG::2012-12-10 17:18:13,991::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' for lock type 'exclusive' >Thread-117::DEBUG::2012-12-10 17:18:13,991::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' is free. Now locking as 'exclusive' (1 active user) >Thread-117::DEBUG::2012-12-10 17:18:13,991::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.5838074b-e545-49d3-a4c8-da1ecd091999`ReqID=`6de16787-cb30-4784-b8fb-3fca915475c9`::Granted request >Thread-117::DEBUG::2012-12-10 17:18:13,991::task::817::TaskManager.Task::(resourceAcquired) Task=`350f3004-f0ba-4399-af6a-95b7daa28ab7`::_resourcesAcquired: Storage.5838074b-e545-49d3-a4c8-da1ecd091999 (exclusive) >Thread-117::DEBUG::2012-12-10 17:18:13,991::task::978::TaskManager.Task::(_decref) Task=`350f3004-f0ba-4399-af6a-95b7daa28ab7`::ref 1 aborting False >Thread-117::INFO::2012-12-10 17:18:13,991::sp::986::Storage.StoragePool::(activateSD) sdUUID=5838074b-e545-49d3-a4c8-da1ecd091999 spUUID=6011f774-ac87-4e98-a0c2-52860f84deef >Thread-117::INFO::2012-12-10 17:18:13,992::logUtils::39::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-117::DEBUG::2012-12-10 17:18:13,992::task::1172::TaskManager.Task::(prepare) Task=`350f3004-f0ba-4399-af6a-95b7daa28ab7`::finished: None >Thread-117::DEBUG::2012-12-10 17:18:13,992::task::588::TaskManager.Task::(_updateState) Task=`350f3004-f0ba-4399-af6a-95b7daa28ab7`::moving from state preparing -> state finished >Thread-117::DEBUG::2012-12-10 17:18:13,992::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>, 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999': < ResourceRef 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999', isValid: 'True' obj: 'None'>} >Thread-117::DEBUG::2012-12-10 17:18:13,992::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-117::DEBUG::2012-12-10 17:18:13,992::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-117::DEBUG::2012-12-10 17:18:13,993::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-117::DEBUG::2012-12-10 17:18:13,993::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-117::DEBUG::2012-12-10 17:18:13,993::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-117::DEBUG::2012-12-10 17:18:13,993::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' >Thread-117::DEBUG::2012-12-10 17:18:13,993::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' (0 active users) >Thread-117::DEBUG::2012-12-10 17:18:13,993::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999' is free, finding out if anyone is waiting for it. >Thread-117::DEBUG::2012-12-10 17:18:13,993::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5838074b-e545-49d3-a4c8-da1ecd091999', Clearing records. >Thread-117::DEBUG::2012-12-10 17:18:13,994::task::978::TaskManager.Task::(_decref) Task=`350f3004-f0ba-4399-af6a-95b7daa28ab7`::ref 0 aborting False >Thread-98::DEBUG::2012-12-10 17:18:21,562::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386 >Thread-98::DEBUG::2012-12-10 17:18:21,565::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-98::ERROR::2012-12-10 17:18:21,579::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `5838074b-e545-49d3-a4c8-da1ecd091999` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386: device is busy\n') >Thread-98::DEBUG::2012-12-10 17:18:21,580::domainMonitor::130::Storage.DomainMonitor::(_monitorDomain) Domain `5838074b-e545-49d3-a4c8-da1ecd091999` changed its status to Invalid >Thread-118::DEBUG::2012-12-10 17:18:21,580::misc::1175::Event.Storage.DomainMonitor.onDomainConnectivityStateChange::(_emit) Emitting event >Thread-118::DEBUG::2012-12-10 17:18:21,581::misc::1186::Event.Storage.DomainMonitor.onDomainConnectivityStateChange::(_emit) Calling registered method `<functools.partial object at 0x7f81100e2998>` >Thread-118::DEBUG::2012-12-10 17:18:21,581::misc::1196::Event.Storage.DomainMonitor.onDomainConnectivityStateChange::(_emit) Event emitted >Thread-120::DEBUG::2012-12-10 17:18:24,071::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-120::DEBUG::2012-12-10 17:18:24,071::task::588::TaskManager.Task::(_updateState) Task=`631eea03-ba01-4f17-b7d3-21b66af1ec3b`::moving from state init -> state preparing >Thread-120::INFO::2012-12-10 17:18:24,071::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-120::INFO::2012-12-10 17:18:24,072::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-120::DEBUG::2012-12-10 17:18:24,072::task::1172::TaskManager.Task::(prepare) Task=`631eea03-ba01-4f17-b7d3-21b66af1ec3b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-120::DEBUG::2012-12-10 17:18:24,072::task::588::TaskManager.Task::(_updateState) Task=`631eea03-ba01-4f17-b7d3-21b66af1ec3b`::moving from state preparing -> state finished >Thread-120::DEBUG::2012-12-10 17:18:24,072::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-120::DEBUG::2012-12-10 17:18:24,072::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-120::DEBUG::2012-12-10 17:18:24,072::task::978::TaskManager.Task::(_decref) Task=`631eea03-ba01-4f17-b7d3-21b66af1ec3b`::ref 0 aborting False >Thread-121::DEBUG::2012-12-10 17:18:24,113::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-121::DEBUG::2012-12-10 17:18:24,113::task::588::TaskManager.Task::(_updateState) Task=`8b9d25bd-9b77-4ede-aa74-095bac348bbe`::moving from state init -> state preparing >Thread-121::INFO::2012-12-10 17:18:24,114::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-121::DEBUG::2012-12-10 17:18:24,114::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`8bd7769c-5643-44d8-acd0-cb9fe1358074`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-121::DEBUG::2012-12-10 17:18:24,114::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'shared' >Thread-121::DEBUG::2012-12-10 17:18:24,114::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'shared' (1 active user) >Thread-121::DEBUG::2012-12-10 17:18:24,115::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`8bd7769c-5643-44d8-acd0-cb9fe1358074`::Granted request >Thread-121::DEBUG::2012-12-10 17:18:24,115::task::817::TaskManager.Task::(resourceAcquired) Task=`8b9d25bd-9b77-4ede-aa74-095bac348bbe`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (shared) >Thread-121::DEBUG::2012-12-10 17:18:24,115::task::978::TaskManager.Task::(_decref) Task=`8b9d25bd-9b77-4ede-aa74-095bac348bbe`::ref 1 aborting False >Thread-121::INFO::2012-12-10 17:18:24,119::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-121::DEBUG::2012-12-10 17:18:24,120::task::1172::TaskManager.Task::(prepare) Task=`8b9d25bd-9b77-4ede-aa74-095bac348bbe`::finished: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-121::DEBUG::2012-12-10 17:18:24,120::task::588::TaskManager.Task::(_updateState) Task=`8b9d25bd-9b77-4ede-aa74-095bac348bbe`::moving from state preparing -> state finished >Thread-121::DEBUG::2012-12-10 17:18:24,120::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>} >Thread-121::DEBUG::2012-12-10 17:18:24,120::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-121::DEBUG::2012-12-10 17:18:24,120::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-121::DEBUG::2012-12-10 17:18:24,120::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-121::DEBUG::2012-12-10 17:18:24,121::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-121::DEBUG::2012-12-10 17:18:24,121::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-121::DEBUG::2012-12-10 17:18:24,121::task::978::TaskManager.Task::(_decref) Task=`8b9d25bd-9b77-4ede-aa74-095bac348bbe`::ref 0 aborting False >Thread-98::DEBUG::2012-12-10 17:18:31,587::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386 >Thread-98::DEBUG::2012-12-10 17:18:31,591::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-98::ERROR::2012-12-10 17:18:31,611::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `5838074b-e545-49d3-a4c8-da1ecd091999` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386: device is busy\n') >Thread-122::DEBUG::2012-12-10 17:18:34,176::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-122::DEBUG::2012-12-10 17:18:34,177::task::588::TaskManager.Task::(_updateState) Task=`ff21fe40-ea7a-448e-ab44-37614bb294a2`::moving from state init -> state preparing >Thread-122::INFO::2012-12-10 17:18:34,177::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-122::INFO::2012-12-10 17:18:34,177::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-122::DEBUG::2012-12-10 17:18:34,178::task::1172::TaskManager.Task::(prepare) Task=`ff21fe40-ea7a-448e-ab44-37614bb294a2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-122::DEBUG::2012-12-10 17:18:34,178::task::588::TaskManager.Task::(_updateState) Task=`ff21fe40-ea7a-448e-ab44-37614bb294a2`::moving from state preparing -> state finished >Thread-122::DEBUG::2012-12-10 17:18:34,178::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2012-12-10 17:18:34,178::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2012-12-10 17:18:34,178::task::978::TaskManager.Task::(_decref) Task=`ff21fe40-ea7a-448e-ab44-37614bb294a2`::ref 0 aborting False >Thread-123::DEBUG::2012-12-10 17:18:34,205::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-123::DEBUG::2012-12-10 17:18:34,205::task::588::TaskManager.Task::(_updateState) Task=`0ca5e0f4-90ea-42b9-b79d-3324db8c76c6`::moving from state init -> state preparing >Thread-123::INFO::2012-12-10 17:18:34,205::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-123::DEBUG::2012-12-10 17:18:34,205::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`d3c40663-a460-44e7-84d0-8a90c3e0d5b8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-123::DEBUG::2012-12-10 17:18:34,205::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'shared' >Thread-123::DEBUG::2012-12-10 17:18:34,206::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'shared' (1 active user) >Thread-123::DEBUG::2012-12-10 17:18:34,206::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`d3c40663-a460-44e7-84d0-8a90c3e0d5b8`::Granted request >Thread-123::DEBUG::2012-12-10 17:18:34,206::task::817::TaskManager.Task::(resourceAcquired) Task=`0ca5e0f4-90ea-42b9-b79d-3324db8c76c6`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (shared) >Thread-123::DEBUG::2012-12-10 17:18:34,206::task::978::TaskManager.Task::(_decref) Task=`0ca5e0f4-90ea-42b9-b79d-3324db8c76c6`::ref 1 aborting False >Thread-123::INFO::2012-12-10 17:18:34,210::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-123::DEBUG::2012-12-10 17:18:34,210::task::1172::TaskManager.Task::(prepare) Task=`0ca5e0f4-90ea-42b9-b79d-3324db8c76c6`::finished: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-123::DEBUG::2012-12-10 17:18:34,210::task::588::TaskManager.Task::(_updateState) Task=`0ca5e0f4-90ea-42b9-b79d-3324db8c76c6`::moving from state preparing -> state finished >Thread-123::DEBUG::2012-12-10 17:18:34,211::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>} >Thread-123::DEBUG::2012-12-10 17:18:34,211::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-123::DEBUG::2012-12-10 17:18:34,211::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-123::DEBUG::2012-12-10 17:18:34,211::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-123::DEBUG::2012-12-10 17:18:34,211::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-123::DEBUG::2012-12-10 17:18:34,211::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-123::DEBUG::2012-12-10 17:18:34,212::task::978::TaskManager.Task::(_decref) Task=`0ca5e0f4-90ea-42b9-b79d-3324db8c76c6`::ref 0 aborting False >Thread-98::DEBUG::2012-12-10 17:18:41,617::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386 >Thread-98::DEBUG::2012-12-10 17:18:41,620::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-98::ERROR::2012-12-10 17:18:41,633::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `5838074b-e545-49d3-a4c8-da1ecd091999` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386: device is busy\n') >Thread-124::DEBUG::2012-12-10 17:18:44,267::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-124::DEBUG::2012-12-10 17:18:44,267::task::588::TaskManager.Task::(_updateState) Task=`1bba4722-b926-441c-95b2-58ca7e44c816`::moving from state init -> state preparing >Thread-124::INFO::2012-12-10 17:18:44,267::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-124::INFO::2012-12-10 17:18:44,268::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-124::DEBUG::2012-12-10 17:18:44,268::task::1172::TaskManager.Task::(prepare) Task=`1bba4722-b926-441c-95b2-58ca7e44c816`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-124::DEBUG::2012-12-10 17:18:44,268::task::588::TaskManager.Task::(_updateState) Task=`1bba4722-b926-441c-95b2-58ca7e44c816`::moving from state preparing -> state finished >Thread-124::DEBUG::2012-12-10 17:18:44,268::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-124::DEBUG::2012-12-10 17:18:44,268::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-124::DEBUG::2012-12-10 17:18:44,268::task::978::TaskManager.Task::(_decref) Task=`1bba4722-b926-441c-95b2-58ca7e44c816`::ref 0 aborting False >Thread-125::DEBUG::2012-12-10 17:18:44,329::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-125::DEBUG::2012-12-10 17:18:44,329::task::588::TaskManager.Task::(_updateState) Task=`dd1dca85-8cab-4997-8dd1-16e362aedccd`::moving from state init -> state preparing >Thread-125::INFO::2012-12-10 17:18:44,329::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-125::DEBUG::2012-12-10 17:18:44,330::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`902ede83-970b-45d9-9ef5-8559853c0ba9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-125::DEBUG::2012-12-10 17:18:44,330::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'shared' >Thread-125::DEBUG::2012-12-10 17:18:44,330::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'shared' (1 active user) >Thread-125::DEBUG::2012-12-10 17:18:44,330::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`902ede83-970b-45d9-9ef5-8559853c0ba9`::Granted request >Thread-125::DEBUG::2012-12-10 17:18:44,330::task::817::TaskManager.Task::(resourceAcquired) Task=`dd1dca85-8cab-4997-8dd1-16e362aedccd`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (shared) >Thread-125::DEBUG::2012-12-10 17:18:44,331::task::978::TaskManager.Task::(_decref) Task=`dd1dca85-8cab-4997-8dd1-16e362aedccd`::ref 1 aborting False >Thread-125::INFO::2012-12-10 17:18:44,334::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-125::DEBUG::2012-12-10 17:18:44,335::task::1172::TaskManager.Task::(prepare) Task=`dd1dca85-8cab-4997-8dd1-16e362aedccd`::finished: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-125::DEBUG::2012-12-10 17:18:44,335::task::588::TaskManager.Task::(_updateState) Task=`dd1dca85-8cab-4997-8dd1-16e362aedccd`::moving from state preparing -> state finished >Thread-125::DEBUG::2012-12-10 17:18:44,335::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>} >Thread-125::DEBUG::2012-12-10 17:18:44,335::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-125::DEBUG::2012-12-10 17:18:44,335::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-125::DEBUG::2012-12-10 17:18:44,335::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-125::DEBUG::2012-12-10 17:18:44,336::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-125::DEBUG::2012-12-10 17:18:44,336::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-125::DEBUG::2012-12-10 17:18:44,336::task::978::TaskManager.Task::(_decref) Task=`dd1dca85-8cab-4997-8dd1-16e362aedccd`::ref 0 aborting False >Thread-98::DEBUG::2012-12-10 17:18:51,640::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386 >Thread-98::DEBUG::2012-12-10 17:18:51,644::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-98::ERROR::2012-12-10 17:18:51,662::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `5838074b-e545-49d3-a4c8-da1ecd091999` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386: device is busy\n') >Thread-126::DEBUG::2012-12-10 17:18:54,444::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-126::DEBUG::2012-12-10 17:18:54,444::task::588::TaskManager.Task::(_updateState) Task=`2cda6ce0-ada9-41c0-a57c-ece40f7e7678`::moving from state init -> state preparing >Thread-126::INFO::2012-12-10 17:18:54,444::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-126::INFO::2012-12-10 17:18:54,445::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-126::DEBUG::2012-12-10 17:18:54,445::task::1172::TaskManager.Task::(prepare) Task=`2cda6ce0-ada9-41c0-a57c-ece40f7e7678`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-126::DEBUG::2012-12-10 17:18:54,445::task::588::TaskManager.Task::(_updateState) Task=`2cda6ce0-ada9-41c0-a57c-ece40f7e7678`::moving from state preparing -> state finished >Thread-126::DEBUG::2012-12-10 17:18:54,445::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2012-12-10 17:18:54,445::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2012-12-10 17:18:54,445::task::978::TaskManager.Task::(_decref) Task=`2cda6ce0-ada9-41c0-a57c-ece40f7e7678`::ref 0 aborting False >Thread-127::DEBUG::2012-12-10 17:18:54,482::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-127::DEBUG::2012-12-10 17:18:54,482::task::588::TaskManager.Task::(_updateState) Task=`48ab8657-fcdb-454b-a65a-ac26a0449efd`::moving from state init -> state preparing >Thread-127::INFO::2012-12-10 17:18:54,482::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-127::DEBUG::2012-12-10 17:18:54,482::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`eb607be2-0ee1-44bc-9fe1-f1cb19d96f86`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-127::DEBUG::2012-12-10 17:18:54,483::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'shared' >Thread-127::DEBUG::2012-12-10 17:18:54,483::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'shared' (1 active user) >Thread-127::DEBUG::2012-12-10 17:18:54,483::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`eb607be2-0ee1-44bc-9fe1-f1cb19d96f86`::Granted request >Thread-127::DEBUG::2012-12-10 17:18:54,483::task::817::TaskManager.Task::(resourceAcquired) Task=`48ab8657-fcdb-454b-a65a-ac26a0449efd`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (shared) >Thread-127::DEBUG::2012-12-10 17:18:54,483::task::978::TaskManager.Task::(_decref) Task=`48ab8657-fcdb-454b-a65a-ac26a0449efd`::ref 1 aborting False >Thread-127::INFO::2012-12-10 17:18:54,487::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-127::DEBUG::2012-12-10 17:18:54,487::task::1172::TaskManager.Task::(prepare) Task=`48ab8657-fcdb-454b-a65a-ac26a0449efd`::finished: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-127::DEBUG::2012-12-10 17:18:54,488::task::588::TaskManager.Task::(_updateState) Task=`48ab8657-fcdb-454b-a65a-ac26a0449efd`::moving from state preparing -> state finished >Thread-127::DEBUG::2012-12-10 17:18:54,488::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>} >Thread-127::DEBUG::2012-12-10 17:18:54,488::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-127::DEBUG::2012-12-10 17:18:54,488::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-127::DEBUG::2012-12-10 17:18:54,488::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-127::DEBUG::2012-12-10 17:18:54,488::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-127::DEBUG::2012-12-10 17:18:54,489::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-127::DEBUG::2012-12-10 17:18:54,489::task::978::TaskManager.Task::(_decref) Task=`48ab8657-fcdb-454b-a65a-ac26a0449efd`::ref 0 aborting False >Thread-98::DEBUG::2012-12-10 17:19:01,668::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386 >Thread-98::DEBUG::2012-12-10 17:19:01,671::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-98::ERROR::2012-12-10 17:19:01,684::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `5838074b-e545-49d3-a4c8-da1ecd091999` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386: device is busy\n') >Thread-128::DEBUG::2012-12-10 17:19:04,537::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-128::DEBUG::2012-12-10 17:19:04,537::task::588::TaskManager.Task::(_updateState) Task=`f53479fb-271f-4e75-85e6-feef946187c2`::moving from state init -> state preparing >Thread-128::INFO::2012-12-10 17:19:04,538::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-128::INFO::2012-12-10 17:19:04,538::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-128::DEBUG::2012-12-10 17:19:04,538::task::1172::TaskManager.Task::(prepare) Task=`f53479fb-271f-4e75-85e6-feef946187c2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-128::DEBUG::2012-12-10 17:19:04,538::task::588::TaskManager.Task::(_updateState) Task=`f53479fb-271f-4e75-85e6-feef946187c2`::moving from state preparing -> state finished >Thread-128::DEBUG::2012-12-10 17:19:04,538::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2012-12-10 17:19:04,538::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2012-12-10 17:19:04,539::task::978::TaskManager.Task::(_decref) Task=`f53479fb-271f-4e75-85e6-feef946187c2`::ref 0 aborting False >Thread-129::DEBUG::2012-12-10 17:19:04,565::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-129::DEBUG::2012-12-10 17:19:04,565::task::588::TaskManager.Task::(_updateState) Task=`652a95e7-462d-4d82-ab00-f53b70c19fa7`::moving from state init -> state preparing >Thread-129::INFO::2012-12-10 17:19:04,565::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-129::DEBUG::2012-12-10 17:19:04,566::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`b24a52d3-0660-4214-94d5-1e7d6339d5fb`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-129::DEBUG::2012-12-10 17:19:04,566::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'shared' >Thread-129::DEBUG::2012-12-10 17:19:04,566::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'shared' (1 active user) >Thread-129::DEBUG::2012-12-10 17:19:04,566::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`b24a52d3-0660-4214-94d5-1e7d6339d5fb`::Granted request >Thread-129::DEBUG::2012-12-10 17:19:04,567::task::817::TaskManager.Task::(resourceAcquired) Task=`652a95e7-462d-4d82-ab00-f53b70c19fa7`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (shared) >Thread-129::DEBUG::2012-12-10 17:19:04,567::task::978::TaskManager.Task::(_decref) Task=`652a95e7-462d-4d82-ab00-f53b70c19fa7`::ref 1 aborting False >Thread-129::INFO::2012-12-10 17:19:04,571::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-129::DEBUG::2012-12-10 17:19:04,571::task::1172::TaskManager.Task::(prepare) Task=`652a95e7-462d-4d82-ab00-f53b70c19fa7`::finished: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-129::DEBUG::2012-12-10 17:19:04,571::task::588::TaskManager.Task::(_updateState) Task=`652a95e7-462d-4d82-ab00-f53b70c19fa7`::moving from state preparing -> state finished >Thread-129::DEBUG::2012-12-10 17:19:04,571::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>} >Thread-129::DEBUG::2012-12-10 17:19:04,571::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-129::DEBUG::2012-12-10 17:19:04,571::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-129::DEBUG::2012-12-10 17:19:04,572::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-129::DEBUG::2012-12-10 17:19:04,572::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-129::DEBUG::2012-12-10 17:19:04,572::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-129::DEBUG::2012-12-10 17:19:04,572::task::978::TaskManager.Task::(_decref) Task=`652a95e7-462d-4d82-ab00-f53b70c19fa7`::ref 0 aborting False >Thread-98::DEBUG::2012-12-10 17:19:11,689::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386 >Thread-98::DEBUG::2012-12-10 17:19:11,693::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-98::ERROR::2012-12-10 17:19:11,706::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `5838074b-e545-49d3-a4c8-da1ecd091999` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386: device is busy\n') >Thread-130::DEBUG::2012-12-10 17:19:14,634::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-130::DEBUG::2012-12-10 17:19:14,634::task::588::TaskManager.Task::(_updateState) Task=`91370983-0634-42c2-b369-ad1d0cbad6cc`::moving from state init -> state preparing >Thread-130::INFO::2012-12-10 17:19:14,634::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-130::INFO::2012-12-10 17:19:14,635::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-130::DEBUG::2012-12-10 17:19:14,635::task::1172::TaskManager.Task::(prepare) Task=`91370983-0634-42c2-b369-ad1d0cbad6cc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-130::DEBUG::2012-12-10 17:19:14,635::task::588::TaskManager.Task::(_updateState) Task=`91370983-0634-42c2-b369-ad1d0cbad6cc`::moving from state preparing -> state finished >Thread-130::DEBUG::2012-12-10 17:19:14,635::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2012-12-10 17:19:14,635::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2012-12-10 17:19:14,635::task::978::TaskManager.Task::(_decref) Task=`91370983-0634-42c2-b369-ad1d0cbad6cc`::ref 0 aborting False >Thread-131::DEBUG::2012-12-10 17:19:14,680::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-131::DEBUG::2012-12-10 17:19:14,680::task::588::TaskManager.Task::(_updateState) Task=`51ea7b00-99b8-4216-82b6-9c5029bfda8b`::moving from state init -> state preparing >Thread-131::INFO::2012-12-10 17:19:14,680::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-131::DEBUG::2012-12-10 17:19:14,680::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`66f6cbee-0d6c-43c1-bf73-f0ce4159c4dd`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-131::DEBUG::2012-12-10 17:19:14,681::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'shared' >Thread-131::DEBUG::2012-12-10 17:19:14,681::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'shared' (1 active user) >Thread-131::DEBUG::2012-12-10 17:19:14,681::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`66f6cbee-0d6c-43c1-bf73-f0ce4159c4dd`::Granted request >Thread-131::DEBUG::2012-12-10 17:19:14,681::task::817::TaskManager.Task::(resourceAcquired) Task=`51ea7b00-99b8-4216-82b6-9c5029bfda8b`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (shared) >Thread-131::DEBUG::2012-12-10 17:19:14,681::task::978::TaskManager.Task::(_decref) Task=`51ea7b00-99b8-4216-82b6-9c5029bfda8b`::ref 1 aborting False >Thread-131::INFO::2012-12-10 17:19:14,685::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-131::DEBUG::2012-12-10 17:19:14,685::task::1172::TaskManager.Task::(prepare) Task=`51ea7b00-99b8-4216-82b6-9c5029bfda8b`::finished: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-131::DEBUG::2012-12-10 17:19:14,686::task::588::TaskManager.Task::(_updateState) Task=`51ea7b00-99b8-4216-82b6-9c5029bfda8b`::moving from state preparing -> state finished >Thread-131::DEBUG::2012-12-10 17:19:14,686::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>} >Thread-131::DEBUG::2012-12-10 17:19:14,686::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-131::DEBUG::2012-12-10 17:19:14,686::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-131::DEBUG::2012-12-10 17:19:14,686::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-131::DEBUG::2012-12-10 17:19:14,686::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-131::DEBUG::2012-12-10 17:19:14,686::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-131::DEBUG::2012-12-10 17:19:14,687::task::978::TaskManager.Task::(_decref) Task=`51ea7b00-99b8-4216-82b6-9c5029bfda8b`::ref 0 aborting False >Thread-98::DEBUG::2012-12-10 17:19:21,710::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386 >Thread-98::DEBUG::2012-12-10 17:19:21,714::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-98::ERROR::2012-12-10 17:19:21,727::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `5838074b-e545-49d3-a4c8-da1ecd091999` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386: device is busy\n') >Thread-132::DEBUG::2012-12-10 17:19:24,733::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-132::DEBUG::2012-12-10 17:19:24,734::task::588::TaskManager.Task::(_updateState) Task=`75ed0520-3a69-4151-9334-df29f1da678d`::moving from state init -> state preparing >Thread-132::INFO::2012-12-10 17:19:24,734::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-132::INFO::2012-12-10 17:19:24,734::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-132::DEBUG::2012-12-10 17:19:24,734::task::1172::TaskManager.Task::(prepare) Task=`75ed0520-3a69-4151-9334-df29f1da678d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-132::DEBUG::2012-12-10 17:19:24,734::task::588::TaskManager.Task::(_updateState) Task=`75ed0520-3a69-4151-9334-df29f1da678d`::moving from state preparing -> state finished >Thread-132::DEBUG::2012-12-10 17:19:24,734::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2012-12-10 17:19:24,735::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2012-12-10 17:19:24,735::task::978::TaskManager.Task::(_decref) Task=`75ed0520-3a69-4151-9334-df29f1da678d`::ref 0 aborting False >Thread-133::DEBUG::2012-12-10 17:19:24,780::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-133::DEBUG::2012-12-10 17:19:24,780::task::588::TaskManager.Task::(_updateState) Task=`10b40ad0-4a77-4be5-9551-86eb572c5d69`::moving from state init -> state preparing >Thread-133::INFO::2012-12-10 17:19:24,780::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-133::DEBUG::2012-12-10 17:19:24,780::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`7d0ba6b9-a36f-4af8-9e3e-caee736e80d6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-133::DEBUG::2012-12-10 17:19:24,781::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' for lock type 'shared' >Thread-133::DEBUG::2012-12-10 17:19:24,781::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free. Now locking as 'shared' (1 active user) >Thread-133::DEBUG::2012-12-10 17:19:24,781::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.6011f774-ac87-4e98-a0c2-52860f84deef`ReqID=`7d0ba6b9-a36f-4af8-9e3e-caee736e80d6`::Granted request >Thread-133::DEBUG::2012-12-10 17:19:24,781::task::817::TaskManager.Task::(resourceAcquired) Task=`10b40ad0-4a77-4be5-9551-86eb572c5d69`::_resourcesAcquired: Storage.6011f774-ac87-4e98-a0c2-52860f84deef (shared) >Thread-133::DEBUG::2012-12-10 17:19:24,781::task::978::TaskManager.Task::(_decref) Task=`10b40ad0-4a77-4be5-9551-86eb572c5d69`::ref 1 aborting False >Thread-133::INFO::2012-12-10 17:19:24,787::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-133::DEBUG::2012-12-10 17:19:24,787::task::1172::TaskManager.Task::(prepare) Task=`10b40ad0-4a77-4be5-9551-86eb572c5d69`::finished: {'info': {'spm_id': 1, 'master_uuid': '5838074b-e545-49d3-a4c8-da1ecd091999', 'name': 'nfsStorageTest', 'version': '3', 'domains': '5838074b-e545-49d3-a4c8-da1ecd091999:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'5838074b-e545-49d3-a4c8-da1ecd091999': {'status': 'Active', 'alerts': []}}} >Thread-133::DEBUG::2012-12-10 17:19:24,787::task::588::TaskManager.Task::(_updateState) Task=`10b40ad0-4a77-4be5-9551-86eb572c5d69`::moving from state preparing -> state finished >Thread-133::DEBUG::2012-12-10 17:19:24,787::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6011f774-ac87-4e98-a0c2-52860f84deef': < ResourceRef 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', isValid: 'True' obj: 'None'>} >Thread-133::DEBUG::2012-12-10 17:19:24,787::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2012-12-10 17:19:24,787::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' >Thread-133::DEBUG::2012-12-10 17:19:24,788::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' (0 active users) >Thread-133::DEBUG::2012-12-10 17:19:24,788::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef' is free, finding out if anyone is waiting for it. >Thread-133::DEBUG::2012-12-10 17:19:24,788::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6011f774-ac87-4e98-a0c2-52860f84deef', Clearing records. >Thread-133::DEBUG::2012-12-10 17:19:24,788::task::978::TaskManager.Task::(_decref) Task=`10b40ad0-4a77-4be5-9551-86eb572c5d69`::ref 0 aborting False >Thread-98::DEBUG::2012-12-10 17:19:31,732::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386 >Thread-98::DEBUG::2012-12-10 17:19:31,735::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386' (cwd None) >Thread-98::ERROR::2012-12-10 17:19:31,749::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `5838074b-e545-49d3-a4c8-da1ecd091999` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386: device is busy\n') >MainThread::INFO::2012-12-10 17:19:32,754::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-10 17:19:32,755::task::588::TaskManager.Task::(_updateState) Task=`37a5bfb2-6a64-44a9-8823-5f1c9ae4fe8a`::moving from state init -> state preparing >MainThread::INFO::2012-12-10 17:19:32,755::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-10 17:19:32,756::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-10 17:19:32,796::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::DEBUG::2012-12-10 17:19:32,796::clientIF::180::vds::(prepareForShutdown) cannot run prepareForShutdown concurrently >MainThread::INFO::2012-12-10 17:19:32,796::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-10 17:19:32,797::task::1172::TaskManager.Task::(prepare) Task=`37a5bfb2-6a64-44a9-8823-5f1c9ae4fe8a`::finished: None >MainThread::DEBUG::2012-12-10 17:19:32,797::task::588::TaskManager.Task::(_updateState) Task=`37a5bfb2-6a64-44a9-8823-5f1c9ae4fe8a`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-10 17:19:32,797::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-10 17:19:32,797::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-10 17:19:32,797::task::978::TaskManager.Task::(_decref) Task=`37a5bfb2-6a64-44a9-8823-5f1c9ae4fe8a`::ref 0 aborting False >MainThread::INFO::2012-12-10 17:19:32,797::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 14 other threads... >MainThread::INFO::2012-12-10 17:19:32,798::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140193662105344)> >MainThread::INFO::2012-12-10 17:19:32,798::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140193034139392)> >MainThread::INFO::2012-12-10 17:19:32,798::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140193514649344)> >MainThread::INFO::2012-12-10 17:19:32,798::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140193539827456)> >MainThread::INFO::2012-12-10 17:19:32,798::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140192983783168)> >MainThread::INFO::2012-12-10 17:19:32,798::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140193017353984)> >MainThread::INFO::2012-12-10 17:19:32,798::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140193025746688)> >MainThread::INFO::2012-12-10 17:19:32,799::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140193556612864)> >MainThread::INFO::2012-12-10 17:19:32,799::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140193531434752)> >MainThread::INFO::2012-12-10 17:19:32,799::vdsm::78::vds::(run) <_MainThread(MainThread, started 140193892542272)> >MainThread::INFO::2012-12-10 17:19:32,799::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140193548220160)> >MainThread::INFO::2012-12-10 17:19:32,799::vdsm::78::vds::(run) <Thread(Thread-28, started daemon 140192472090368)> >MainThread::INFO::2012-12-10 17:19:32,799::vdsm::78::vds::(run) <WorkerThread(4045044c-0ebd-417b-8fab-40c1acc5b08e, started daemon 140193565005568)> >MainThread::INFO::2012-12-10 17:19:32,799::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140193523042048)> >MainThread::INFO::2012-12-10 17:19:32,799::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-10 17:19:32,799::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140192488875776)> >MainThread::INFO::2012-12-10 17:19:32,881::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-10 17:19:33,045::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-10 17:19:33,045::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-10 17:19:33,070::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 1885' (cwd None) >MainThread::DEBUG::2012-12-10 17:19:33,093::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:19:33,094::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-10 17:19:33,094::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 1bc7e360-6762-4771-8a91-9ab9eb6c9c07 2620' (cwd None) >MainThread::DEBUG::2012-12-10 17:19:33,229::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-10 17:19:33,229::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-10 17:19:33,229::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-10 17:19:33,229::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-10 17:19:33,230::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-10 17:19:33,230::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-10 17:19:33,231::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-10 17:19:35,099::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-10 17:19:35,105::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-10 17:19:35,105::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-10 17:19:35,106::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-10 17:19:35,126::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:19:35,126::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 17:19:35,127::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-10 17:19:35,143::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:19:35,144::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 17:19:35,144::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 17:19:35,144::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-10 17:19:35,160::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:19:35,160::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 17:19:35,160::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-10 17:19:35,176::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::INFO::2012-12-10 17:19:35,178::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-10 17:19:35,178::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-12::DEBUG::2012-12-10 17:19:35,178::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 17:19:35,179::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-10 17:19:35,179::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 17:19:35,180::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-10 17:19:35,188::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-10 17:19:35,191::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:19:35,191::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-10 17:19:35,196::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-10 17:19:35,197::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-10 17:19:35,198::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-10 17:19:35,199::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-10 17:19:37,030::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-10 17:19:37,073::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:19:37,074::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:19:37,074::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:19:37,074::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:19:37,074::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:19:37,075::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:19:37,075::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:19:37,075::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-10 17:19:37,075::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-10 17:19:37,079::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-10 17:19:37,079::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386', '/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257', '/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601'] >Thread-12::DEBUG::2012-12-10 17:19:37,080::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-10 17:19:37,081::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-10 17:19:44,953::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-14::DEBUG::2012-12-10 17:19:44,954::task::588::TaskManager.Task::(_updateState) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::moving from state init -> state preparing >Thread-14::INFO::2012-12-10 17:19:44,954::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='6011f774-ac87-4e98-a0c2-52860f84deef', options=None) >Thread-14::ERROR::2012-12-10 17:19:44,954::task::853::TaskManager.Task::(_setError) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 861, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 38, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 528, in getSpmStatus > pool = self.getPool(spUUID) > File "/usr/share/vdsm/storage/hsm.py", line 265, in getPool > raise se.StoragePoolUnknown(spUUID) >StoragePoolUnknown: Unknown pool id, pool not connected: ('6011f774-ac87-4e98-a0c2-52860f84deef',) >Thread-14::DEBUG::2012-12-10 17:19:44,955::task::872::TaskManager.Task::(_run) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::Task._run: e3bad80c-c99d-4a8d-ad13-3e05e58abc2e ('6011f774-ac87-4e98-a0c2-52860f84deef',) {} failed - stopping task >Thread-14::DEBUG::2012-12-10 17:19:44,956::task::1199::TaskManager.Task::(stop) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::stopping in state preparing (force False) >Thread-14::DEBUG::2012-12-10 17:19:44,956::task::978::TaskManager.Task::(_decref) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::ref 1 aborting True >Thread-14::INFO::2012-12-10 17:19:44,956::task::1157::TaskManager.Task::(prepare) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-14::DEBUG::2012-12-10 17:19:44,956::task::1162::TaskManager.Task::(prepare) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::Prepare: aborted: Unknown pool id, pool not connected >Thread-14::DEBUG::2012-12-10 17:19:44,956::task::978::TaskManager.Task::(_decref) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::ref 0 aborting True >Thread-14::DEBUG::2012-12-10 17:19:44,956::task::913::TaskManager.Task::(_doAbort) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::Task._doAbort: force False >Thread-14::DEBUG::2012-12-10 17:19:44,956::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2012-12-10 17:19:44,957::task::588::TaskManager.Task::(_updateState) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::moving from state preparing -> state aborting >Thread-14::DEBUG::2012-12-10 17:19:44,957::task::537::TaskManager.Task::(__state_aborting) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::_aborting: recover policy none >Thread-14::DEBUG::2012-12-10 17:19:44,957::task::588::TaskManager.Task::(_updateState) Task=`e3bad80c-c99d-4a8d-ad13-3e05e58abc2e`::moving from state aborting -> state failed >Thread-14::DEBUG::2012-12-10 17:19:44,957::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2012-12-10 17:19:44,957::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::ERROR::2012-12-10 17:19:44,957::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('6011f774-ac87-4e98-a0c2-52860f84deef',)", 'code': 309}} >Thread-15::DEBUG::2012-12-10 17:19:45,022::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-15::DEBUG::2012-12-10 17:19:45,023::task::588::TaskManager.Task::(_updateState) Task=`a6b45989-8af3-49d0-b93f-0c1970fc7e30`::moving from state init -> state preparing >Thread-15::INFO::2012-12-10 17:19:45,023::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-15::DEBUG::2012-12-10 17:19:45,023::taskManager::103::TaskManager::(getAllTasksStatuses) Entry. >Thread-15::DEBUG::2012-12-10 17:19:45,023::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {} >Thread-15::INFO::2012-12-10 17:19:45,023::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-15::DEBUG::2012-12-10 17:19:45,023::task::1172::TaskManager.Task::(prepare) Task=`a6b45989-8af3-49d0-b93f-0c1970fc7e30`::finished: {'allTasksStatus': {}} >Thread-15::DEBUG::2012-12-10 17:19:45,023::task::588::TaskManager.Task::(_updateState) Task=`a6b45989-8af3-49d0-b93f-0c1970fc7e30`::moving from state preparing -> state finished >Thread-15::DEBUG::2012-12-10 17:19:45,023::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-15::DEBUG::2012-12-10 17:19:45,024::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-15::DEBUG::2012-12-10 17:19:45,024::task::978::TaskManager.Task::(_decref) Task=`a6b45989-8af3-49d0-b93f-0c1970fc7e30`::ref 0 aborting False >MainThread::INFO::2012-12-10 17:44:55,666::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-10 17:44:55,667::task::588::TaskManager.Task::(_updateState) Task=`c60b6d91-4020-4f2a-b89e-b81eec63c344`::moving from state init -> state preparing >MainThread::INFO::2012-12-10 17:44:55,667::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-10 17:44:55,667::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-10 17:44:55,695::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-10 17:44:55,695::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-10 17:44:55,695::task::1172::TaskManager.Task::(prepare) Task=`c60b6d91-4020-4f2a-b89e-b81eec63c344`::finished: None >MainThread::DEBUG::2012-12-10 17:44:55,695::task::588::TaskManager.Task::(_updateState) Task=`c60b6d91-4020-4f2a-b89e-b81eec63c344`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-10 17:44:55,695::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-10 17:44:55,696::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-10 17:44:55,696::task::978::TaskManager.Task::(_decref) Task=`c60b6d91-4020-4f2a-b89e-b81eec63c344`::ref 0 aborting False >MainThread::INFO::2012-12-10 17:44:55,696::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-10 17:44:55,696::vdsm::78::vds::(run) <_MainThread(MainThread, started 140449547646784)> >MainThread::INFO::2012-12-10 17:44:55,696::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140449317209856)> >MainThread::INFO::2012-12-10 17:44:55,696::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140449020884736)> >MainThread::INFO::2012-12-10 17:44:55,696::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140449004099328)> >MainThread::INFO::2012-12-10 17:44:55,697::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140449029277440)> >MainThread::INFO::2012-12-10 17:44:55,697::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140448425297664)> >MainThread::INFO::2012-12-10 17:44:55,697::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140449037670144)> >MainThread::INFO::2012-12-10 17:44:55,697::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140449046062848)> >MainThread::INFO::2012-12-10 17:44:55,697::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-10 17:44:55,697::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140448408512256)> >MainThread::INFO::2012-12-10 17:44:55,697::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140449287841536)> >MainThread::INFO::2012-12-10 17:44:55,697::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140449054455552)> >MainThread::INFO::2012-12-10 17:44:55,697::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140449012492032)> >MainThread::INFO::2012-12-10 17:44:55,698::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140449279448832)> >MainThread::INFO::2012-12-10 17:44:55,698::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140449271056128)> >MainThread::INFO::2012-12-10 17:44:57,110::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-10 17:44:57,273::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-10 17:44:57,273::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-10 17:44:57,301::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 2737' (cwd None) >MainThread::DEBUG::2012-12-10 17:44:57,310::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 2737: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-10 17:44:57,311::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-10 17:44:57,311::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 9c38b0e3-5ee3-48c9-9067-4c00db291b13 5276' (cwd None) >MainThread::DEBUG::2012-12-10 17:44:57,444::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-10 17:44:57,444::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-10 17:44:57,444::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-10 17:44:57,445::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-10 17:44:57,445::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-10 17:44:57,445::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-10 17:44:57,446::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-10 17:44:59,315::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-10 17:44:59,321::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-10 17:44:59,322::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-10 17:44:59,322::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-10 17:44:59,341::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:44:59,341::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 17:44:59,342::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-10 17:44:59,358::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:44:59,358::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 17:44:59,358::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 17:44:59,359::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-10 17:44:59,373::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:44:59,373::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 17:44:59,374::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-10 17:44:59,388::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:44:59,389::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-10 17:44:59,390::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-10 17:44:59,390::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 17:44:59,390::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-10 17:44:59,391::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 17:44:59,391::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-10 17:44:59,401::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-10 17:44:59,405::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:44:59,405::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-10 17:44:59,409::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-10 17:44:59,410::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-10 17:44:59,412::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-10 17:44:59,413::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-10 17:45:01,242::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-10 17:45:01,297::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:45:01,298::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:01,298::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:01,299::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:01,299::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:01,299::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:01,299::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:01,299::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-10 17:45:01,299::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-10 17:45:01,304::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-10 17:45:01,304::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386', '/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257', '/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601'] >Thread-12::DEBUG::2012-12-10 17:45:01,304::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-10 17:45:01,305::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-10 17:45:01,416::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-10 17:45:01,417::task::588::TaskManager.Task::(_updateState) Task=`8c2d9ba3-e440-4370-9f95-abeeeb806bbc`::moving from state init -> state preparing >MainThread::INFO::2012-12-10 17:45:01,417::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-10 17:45:01,417::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-10 17:45:01,449::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-10 17:45:01,449::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-10 17:45:01,449::task::1172::TaskManager.Task::(prepare) Task=`8c2d9ba3-e440-4370-9f95-abeeeb806bbc`::finished: None >MainThread::DEBUG::2012-12-10 17:45:01,449::task::588::TaskManager.Task::(_updateState) Task=`8c2d9ba3-e440-4370-9f95-abeeeb806bbc`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-10 17:45:01,450::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-10 17:45:01,450::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-10 17:45:01,450::task::978::TaskManager.Task::(_decref) Task=`8c2d9ba3-e440-4370-9f95-abeeeb806bbc`::ref 0 aborting False >MainThread::INFO::2012-12-10 17:45:01,450::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-10 17:45:01,450::vdsm::78::vds::(run) <_MainThread(MainThread, started 139640758093632)> >MainThread::INFO::2012-12-10 17:45:01,450::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139640527652608)> >MainThread::INFO::2012-12-10 17:45:01,450::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-10 17:45:01,450::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139639553767168)> >MainThread::INFO::2012-12-10 17:45:01,451::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139640090638080)> >MainThread::INFO::2012-12-10 17:45:01,451::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139640481498880)> >MainThread::INFO::2012-12-10 17:45:01,451::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139640115816192)> >MainThread::INFO::2012-12-10 17:45:01,451::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139640489891584)> >MainThread::INFO::2012-12-10 17:45:01,451::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139639570552576)> >MainThread::INFO::2012-12-10 17:45:01,451::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139640498284288)> >MainThread::INFO::2012-12-10 17:45:01,451::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139640099030784)> >MainThread::INFO::2012-12-10 17:45:01,451::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139640124208896)> >MainThread::INFO::2012-12-10 17:45:01,452::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139640082245376)> >MainThread::INFO::2012-12-10 17:45:01,452::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139640107423488)> >MainThread::INFO::2012-12-10 17:45:01,452::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139640473106176)> >MainThread::INFO::2012-12-10 17:45:02,743::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-10 17:45:02,905::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-10 17:45:02,906::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-10 17:45:02,936::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 5297' (cwd None) >MainThread::DEBUG::2012-12-10 17:45:02,946::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 5297: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-10 17:45:02,947::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-10 17:45:02,947::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 75633676-88a2-412a-9085-13e47a7994f0 5672' (cwd None) >MainThread::DEBUG::2012-12-10 17:45:03,082::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-10 17:45:03,082::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-10 17:45:03,082::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-10 17:45:03,083::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-10 17:45:03,083::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-10 17:45:03,083::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-10 17:45:03,084::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-10 17:45:04,952::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-10 17:45:04,958::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-10 17:45:04,959::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-10 17:45:04,959::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-10 17:45:04,979::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:45:04,979::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 17:45:04,980::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-10 17:45:04,996::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:45:04,996::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 17:45:04,996::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-10 17:45:04,997::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-10 17:45:05,012::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-10 17:45:05,012::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-10 17:45:05,013::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-10 17:45:05,028::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:45:05,029::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-10 17:45:05,030::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-10 17:45:05,030::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 17:45:05,030::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-10 17:45:05,031::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-10 17:45:05,031::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-10 17:45:05,039::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-10 17:45:05,043::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:45:05,043::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-10 17:45:05,048::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-10 17:45:05,049::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-10 17:45:05,051::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-10 17:45:05,053::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-10 17:45:06,895::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-10 17:45:07,019::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-10 17:45:07,019::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:07,019::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:07,020::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:07,020::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:07,020::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:07,020::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-10 17:45:07,020::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-10 17:45:07,021::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-10 17:45:07,025::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-10 17:45:07,025::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386', '/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257', '/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601'] >Thread-12::DEBUG::2012-12-10 17:45:07,026::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-10 17:45:07,026::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-11 00:40:09,480::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-11 00:40:09,480::task::588::TaskManager.Task::(_updateState) Task=`b30b8439-ec88-4af1-ac7c-085d0c598916`::moving from state init -> state preparing >MainThread::INFO::2012-12-11 00:40:09,481::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-11 00:40:09,481::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-11 00:40:09,506::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-11 00:40:09,506::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-11 00:40:09,506::task::1172::TaskManager.Task::(prepare) Task=`b30b8439-ec88-4af1-ac7c-085d0c598916`::finished: None >MainThread::DEBUG::2012-12-11 00:40:09,506::task::588::TaskManager.Task::(_updateState) Task=`b30b8439-ec88-4af1-ac7c-085d0c598916`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-11 00:40:09,507::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-11 00:40:09,507::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-11 00:40:09,507::task::978::TaskManager.Task::(_decref) Task=`b30b8439-ec88-4af1-ac7c-085d0c598916`::ref 0 aborting False >MainThread::INFO::2012-12-11 00:40:09,507::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-11 00:40:09,507::vdsm::78::vds::(run) <_MainThread(MainThread, started 140664941446976)> >MainThread::INFO::2012-12-11 00:40:09,507::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140664711005952)> >MainThread::INFO::2012-12-11 00:40:09,507::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140664071255808)> >MainThread::INFO::2012-12-11 00:40:09,508::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140664574555904)> >MainThread::INFO::2012-12-11 00:40:09,508::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140664062863104)> >MainThread::INFO::2012-12-11 00:40:09,508::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140664054470400)> >MainThread::INFO::2012-12-11 00:40:09,508::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140664591341312)> >MainThread::INFO::2012-12-11 00:40:09,508::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140664020899584)> >MainThread::INFO::2012-12-11 00:40:09,508::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140664599734016)> >MainThread::INFO::2012-12-11 00:40:09,508::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140664608126720)> >MainThread::INFO::2012-12-11 00:40:09,508::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-11 00:40:09,508::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140663525992192)> >MainThread::INFO::2012-12-11 00:40:09,509::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140664566163200)> >MainThread::INFO::2012-12-11 00:40:09,509::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140664557770496)> >MainThread::INFO::2012-12-11 00:40:09,509::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140664582948608)> >MainThread::INFO::2012-12-11 00:40:10,933::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-11 00:40:11,097::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-11 00:40:11,097::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-11 00:40:11,120::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 5697' (cwd None) >MainThread::DEBUG::2012-12-11 00:40:11,129::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 5697: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-11 00:40:11,130::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-11 00:40:11,130::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 8c306630-052a-41be-8b9a-2ef411955f17 23701' (cwd None) >MainThread::DEBUG::2012-12-11 00:40:11,265::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-11 00:40:11,265::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-11 00:40:11,265::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-11 00:40:11,265::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-11 00:40:11,266::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-11 00:40:11,266::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-11 00:40:11,267::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-11 00:40:13,135::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-11 00:40:13,140::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-11 00:40:13,141::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-11 00:40:13,141::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-11 00:40:13,160::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 00:40:13,160::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 00:40:13,161::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-11 00:40:13,177::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 00:40:13,177::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 00:40:13,178::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 00:40:13,178::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-11 00:40:13,193::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-11 00:40:13,193::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 00:40:13,193::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-11 00:40:13,208::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-11 00:40:13,209::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-11 00:40:13,210::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-11 00:40:13,210::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 00:40:13,211::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-11 00:40:13,211::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 00:40:13,211::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-11 00:40:13,220::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-11 00:40:13,223::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 00:40:13,224::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-11 00:40:13,229::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-11 00:40:13,230::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-11 00:40:13,231::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-11 00:40:13,233::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-11 00:40:15,059::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-11 00:40:15,107::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 00:40:15,107::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 00:40:15,107::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 00:40:15,107::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 00:40:15,108::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 00:40:15,108::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 00:40:15,108::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 00:40:15,108::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-11 00:40:15,108::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-11 00:40:15,113::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-11 00:40:15,113::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386', '/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257', '/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601'] >Thread-12::DEBUG::2012-12-11 00:40:15,113::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-11 00:40:15,114::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-11 05:24:40,818::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-11 05:24:40,819::task::588::TaskManager.Task::(_updateState) Task=`287db326-c21a-4b00-9062-268d2a268e2c`::moving from state init -> state preparing >MainThread::INFO::2012-12-11 05:24:40,819::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-11 05:24:40,820::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-11 05:24:40,844::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-11 05:24:40,844::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-11 05:24:40,844::task::1172::TaskManager.Task::(prepare) Task=`287db326-c21a-4b00-9062-268d2a268e2c`::finished: None >MainThread::DEBUG::2012-12-11 05:24:40,844::task::588::TaskManager.Task::(_updateState) Task=`287db326-c21a-4b00-9062-268d2a268e2c`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-11 05:24:40,844::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-11 05:24:40,845::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-11 05:24:40,845::task::978::TaskManager.Task::(_decref) Task=`287db326-c21a-4b00-9062-268d2a268e2c`::ref 0 aborting False >MainThread::INFO::2012-12-11 05:24:40,845::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 12 other threads... >MainThread::INFO::2012-12-11 05:24:40,845::vdsm::78::vds::(run) <_MainThread(MainThread, started 140244466534208)> >MainThread::INFO::2012-12-11 05:24:40,845::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140244236093184)> >MainThread::INFO::2012-12-11 05:24:40,845::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140244206724864)> >MainThread::INFO::2012-12-11 05:24:40,845::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140243835549440)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140243810371328)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140243818764032)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140243827156736)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140244181546752)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140243793585920)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140243281893120)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140243801978624)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140244198332160)> >MainThread::INFO::2012-12-11 05:24:40,846::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140244189939456)> >MainThread::INFO::2012-12-11 05:24:42,391::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-11 05:24:42,557::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-11 05:24:42,557::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-11 05:24:42,582::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 23722' (cwd None) >MainThread::DEBUG::2012-12-11 05:24:42,592::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 23722: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-11 05:24:42,592::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-11 05:24:42,593::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py f84cec78-19df-4543-a708-4ba1f3d08034 657' (cwd None) >MainThread::DEBUG::2012-12-11 05:24:42,728::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-11 05:24:42,728::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-11 05:24:42,728::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-11 05:24:42,728::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-11 05:24:42,729::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-11 05:24:42,729::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-11 05:24:42,730::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-11 05:24:44,598::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-11 05:24:44,603::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-11 05:24:44,604::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-11 05:24:44,604::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-11 05:24:44,623::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 05:24:44,624::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 05:24:44,625::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-11 05:24:44,642::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 05:24:44,642::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 05:24:44,642::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 05:24:44,643::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-11 05:24:44,658::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-11 05:24:44,658::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 05:24:44,659::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-11 05:24:44,674::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-11 05:24:44,675::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-11 05:24:44,676::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-11 05:24:44,676::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 05:24:44,676::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-11 05:24:44,677::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 05:24:44,677::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-11 05:24:44,686::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-11 05:24:44,689::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 05:24:44,690::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-11 05:24:44,695::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-11 05:24:44,696::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-11 05:24:44,696::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-11 05:24:44,698::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-11 05:24:46,525::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-11 05:24:46,570::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 05:24:46,570::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 05:24:46,570::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 05:24:46,571::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 05:24:46,571::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 05:24:46,571::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 05:24:46,571::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 05:24:46,571::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-11 05:24:46,572::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-11 05:24:46,576::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-11 05:24:46,576::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386', '/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257', '/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601'] >Thread-12::DEBUG::2012-12-11 05:24:46,576::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-11 05:24:46,577::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2012-12-11 10:01:45,776::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-11 10:01:45,776::task::588::TaskManager.Task::(_updateState) Task=`3232a1c1-d53a-4f50-aa40-1038c78239c7`::moving from state init -> state preparing >MainThread::INFO::2012-12-11 10:01:45,777::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-11 10:01:45,777::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-11 10:01:45,801::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-11 10:01:45,801::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-11 10:01:45,802::task::1172::TaskManager.Task::(prepare) Task=`3232a1c1-d53a-4f50-aa40-1038c78239c7`::finished: None >MainThread::DEBUG::2012-12-11 10:01:45,802::task::588::TaskManager.Task::(_updateState) Task=`3232a1c1-d53a-4f50-aa40-1038c78239c7`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-11 10:01:45,802::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-11 10:01:45,802::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-11 10:01:45,802::task::978::TaskManager.Task::(_decref) Task=`3232a1c1-d53a-4f50-aa40-1038c78239c7`::ref 0 aborting False >MainThread::INFO::2012-12-11 10:01:45,802::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <_MainThread(MainThread, started 140508864145216)> >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140508633704192)> >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140508345120512)> >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140508353513216)> >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140508336727808)> >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140508361905920)> >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140508370298624)> >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140507841820416)> >MainThread::INFO::2012-12-11 10:01:45,803::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 140508595943168)> >MainThread::INFO::2012-12-11 10:01:45,804::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140507808249600)> >MainThread::INFO::2012-12-11 10:01:45,804::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140508604335872)> >MainThread::INFO::2012-12-11 10:01:45,804::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140508328335104)> >MainThread::INFO::2012-12-11 10:01:45,804::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140508378691328)> >MainThread::INFO::2012-12-11 10:01:45,804::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-11 10:01:45,804::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140507791464192)> >MainThread::INFO::2012-12-11 10:01:47,443::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-11 10:01:47,609::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-11 10:01:47,609::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-11 10:01:47,635::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 680' (cwd None) >MainThread::DEBUG::2012-12-11 10:01:47,644::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'kill 680: No such process\n'; <rc> = 1 >MainThread::DEBUG::2012-12-11 10:01:47,645::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-11 10:01:47,645::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py a018dba7-bf14-496c-9c75-03fcbc161a8f 9574' (cwd None) >MainThread::DEBUG::2012-12-11 10:01:47,780::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-11 10:01:47,780::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-11 10:01:47,780::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-11 10:01:47,780::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-11 10:01:47,780::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-11 10:01:47,781::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-11 10:01:47,782::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-11 10:01:49,650::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-11 10:01:49,656::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-11 10:01:49,657::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-11 10:01:49,657::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-11 10:01:49,677::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 10:01:49,677::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 10:01:49,678::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-11 10:01:49,694::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 10:01:49,695::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 10:01:49,695::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 10:01:49,695::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-11 10:01:49,710::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-11 10:01:49,711::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 10:01:49,711::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-11 10:01:49,726::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-11 10:01:49,727::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-11 10:01:49,728::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-11 10:01:49,728::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 10:01:49,728::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-11 10:01:49,729::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 10:01:49,729::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-11 10:01:49,737::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-11 10:01:49,742::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 10:01:49,742::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-11 10:01:49,747::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-11 10:01:49,747::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-11 10:01:49,748::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-11 10:01:49,750::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-11 10:01:51,576::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-11 10:01:51,622::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 10:01:51,622::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 10:01:51,623::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 10:01:51,623::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 10:01:51,623::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 10:01:51,623::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 10:01:51,623::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 10:01:51,624::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-11 10:01:51,624::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-11 10:01:51,628::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-11 10:01:51,628::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__53__821386', '/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__10__17__13__55__179257', '/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121210171355962601'] >Thread-12::DEBUG::2012-12-11 10:01:51,628::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-11 10:01:51,629::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-11 11:52:06,028::BindingXMLRPC::859::vds::(wrapper) client [10.35.148.46]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-11 11:52:06,130::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-14::DEBUG::2012-12-11 11:52:06,147::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:52:06,148::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-11 11:52:06,163::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:52:06,163::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-11 11:52:06,179::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:52:06,179::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-11 11:52:06,193::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:52:06,194::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-14::DEBUG::2012-12-11 11:52:06,209::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:52:06,999::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '5.fc18.x86_64', 'buildtime': 1353433208.0, 'version': '3.6.7'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '1.fc18', 'buildtime': '1355100085', 'version': '0.10.2.2'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >MainThread::INFO::2012-12-11 11:52:34,048::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-11 11:52:34,049::task::588::TaskManager.Task::(_updateState) Task=`cb382f87-8379-4b66-a94d-4d36bd9d8b5c`::moving from state init -> state preparing >MainThread::INFO::2012-12-11 11:52:34,049::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-11 11:52:34,050::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-11 11:52:34,077::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-11 11:52:34,078::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-11 11:52:34,078::task::1172::TaskManager.Task::(prepare) Task=`cb382f87-8379-4b66-a94d-4d36bd9d8b5c`::finished: None >MainThread::DEBUG::2012-12-11 11:52:34,078::task::588::TaskManager.Task::(_updateState) Task=`cb382f87-8379-4b66-a94d-4d36bd9d8b5c`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-11 11:52:34,078::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-11 11:52:34,078::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-11 11:52:34,078::task::978::TaskManager.Task::(_decref) Task=`cb382f87-8379-4b66-a94d-4d36bd9d8b5c`::ref 0 aborting False >MainThread::INFO::2012-12-11 11:52:34,078::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 13 other threads... >MainThread::INFO::2012-12-11 11:52:34,079::vdsm::78::vds::(run) <_MainThread(MainThread, started 139895299569472)> >MainThread::INFO::2012-12-11 11:52:34,079::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139895069128448)> >MainThread::INFO::2012-12-11 11:52:34,079::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-11 11:52:34,079::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139894097688320)> >MainThread::INFO::2012-12-11 11:52:34,079::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139894668130048)> >MainThread::INFO::2012-12-11 11:52:34,079::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139894114473728)> >MainThread::INFO::2012-12-11 11:52:34,079::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139894659737344)> >MainThread::INFO::2012-12-11 11:52:34,079::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139894651344640)> >MainThread::INFO::2012-12-11 11:52:34,080::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139895014582016)> >MainThread::INFO::2012-12-11 11:52:34,080::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139894642951936)> >MainThread::INFO::2012-12-11 11:52:34,080::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139895022974720)> >MainThread::INFO::2012-12-11 11:52:34,080::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139894634559232)> >MainThread::INFO::2012-12-11 11:52:34,080::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139894626166528)> >MainThread::INFO::2012-12-11 11:52:34,080::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139895039760128)> >MainThread::INFO::2012-12-11 11:52:34,080::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139895031367424)> >MainThread::INFO::2012-12-11 11:54:44,955::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-11 11:54:46,932::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-11 11:54:46,933::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-11 11:54:46,989::supervdsm::103::SuperVdsmProxy::(_killSupervdsm) Could not kill old Super Vdsm [Errno 2] No such file or directory: '/var/run/vdsm/svdsm.pid' >MainThread::DEBUG::2012-12-11 11:54:46,989::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-11 11:54:46,989::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py a158bb74-65ef-4fdc-9967-67b7022489dd 1303' (cwd None) >MainThread::DEBUG::2012-12-11 11:54:47,157::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-11 11:54:47,157::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-11 11:54:47,157::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-11 11:54:47,158::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-11 11:54:47,158::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-11 11:54:47,158::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-11 11:54:47,159::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-11 11:54:48,995::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-11 11:54:49,001::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-11 11:54:49,002::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-11 11:54:49,002::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-11 11:54:49,024::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 11:54:49,024::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 11:54:49,025::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-11 11:54:49,042::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 11:54:49,042::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 11:54:49,042::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 11:54:49,042::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-11 11:54:49,058::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-11 11:54:49,059::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 11:54:49,059::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-11 11:54:49,076::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-11 11:54:49,076::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-11 11:54:49,077::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-11 11:54:49,077::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 11:54:49,078::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-11 11:54:49,078::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 11:54:49,079::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-11 11:54:49,089::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-11 11:54:49,093::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 11:54:49,094::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-11 11:54:49,101::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-11 11:54:49,102::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-11 11:54:49,102::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-11 11:54:49,105::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-11 11:54:50,929::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-11 11:54:50,973::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 11:54:50,973::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 11:54:50,973::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 11:54:50,974::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 11:54:50,974::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 11:54:50,974::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 11:54:50,974::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 11:54:50,974::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-11 11:54:50,975::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-11 11:54:50,978::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-11 11:54:50,978::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-11 11:54:50,978::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-11 11:54:51,007::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-11 11:55:07,269::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-11 11:55:07,439::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-14::DEBUG::2012-12-11 11:55:07,455::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:55:07,456::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-11 11:55:07,470::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:55:07,471::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-11 11:55:07,486::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:55:07,486::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-11 11:55:07,501::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:55:07,501::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-14::DEBUG::2012-12-11 11:55:07,516::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:55:08,298::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '4.fc18.x86_64', 'buildtime': 1354623171.0, 'version': '3.6.9'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '1.fc18', 'buildtime': '1355100085', 'version': '0.10.2.2'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >Thread-15::DEBUG::2012-12-11 11:55:08,549::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-15::DEBUG::2012-12-11 11:55:08,566::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-15::DEBUG::2012-12-11 11:55:08,582::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 11:55:08,582::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-15::DEBUG::2012-12-11 11:55:08,597::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 11:55:08,598::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-15::DEBUG::2012-12-11 11:55:08,613::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 11:55:08,613::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-15::DEBUG::2012-12-11 11:55:08,628::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 11:55:08,628::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-15::DEBUG::2012-12-11 11:55:08,643::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 11:55:08,644::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '4.fc18.x86_64', 'buildtime': 1354623171.0, 'version': '3.6.9'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '1.fc18', 'buildtime': '1355100085', 'version': '0.10.2.2'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '10.35.18.155'}} >Thread-17::DEBUG::2012-12-11 11:55:11,069::task::588::TaskManager.Task::(_updateState) Task=`ce559624-1bf0-49b2-8b50-95d4e524e6cb`::moving from state init -> state preparing >Thread-17::INFO::2012-12-11 11:55:11,069::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-17::INFO::2012-12-11 11:55:11,070::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-17::DEBUG::2012-12-11 11:55:11,070::task::1172::TaskManager.Task::(prepare) Task=`ce559624-1bf0-49b2-8b50-95d4e524e6cb`::finished: {} >Thread-17::DEBUG::2012-12-11 11:55:11,070::task::588::TaskManager.Task::(_updateState) Task=`ce559624-1bf0-49b2-8b50-95d4e524e6cb`::moving from state preparing -> state finished >Thread-17::DEBUG::2012-12-11 11:55:11,070::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2012-12-11 11:55:11,070::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2012-12-11 11:55:11,070::task::978::TaskManager.Task::(_decref) Task=`ce559624-1bf0-49b2-8b50-95d4e524e6cb`::ref 0 aborting False >Thread-21::DEBUG::2012-12-11 11:55:16,684::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-21::DEBUG::2012-12-11 11:55:16,685::task::588::TaskManager.Task::(_updateState) Task=`ddd0ff9b-ff9c-4e8f-a8f6-5de2dd06e143`::moving from state init -> state preparing >Thread-21::INFO::2012-12-11 11:55:16,685::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-21::INFO::2012-12-11 11:55:16,685::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-21::DEBUG::2012-12-11 11:55:16,685::task::1172::TaskManager.Task::(prepare) Task=`ddd0ff9b-ff9c-4e8f-a8f6-5de2dd06e143`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-21::DEBUG::2012-12-11 11:55:16,685::task::588::TaskManager.Task::(_updateState) Task=`ddd0ff9b-ff9c-4e8f-a8f6-5de2dd06e143`::moving from state preparing -> state finished >Thread-21::DEBUG::2012-12-11 11:55:16,685::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-21::DEBUG::2012-12-11 11:55:16,686::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-21::DEBUG::2012-12-11 11:55:16,686::task::978::TaskManager.Task::(_decref) Task=`ddd0ff9b-ff9c-4e8f-a8f6-5de2dd06e143`::ref 0 aborting False >Thread-22::DEBUG::2012-12-11 11:55:16,754::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-22::DEBUG::2012-12-11 11:55:16,754::task::588::TaskManager.Task::(_updateState) Task=`c60a7cf8-8d18-468a-a069-972307c79818`::moving from state init -> state preparing >Thread-22::INFO::2012-12-11 11:55:16,754::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-22::DEBUG::2012-12-11 11:55:16,758::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873 /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-22::DEBUG::2012-12-11 11:55:17,009::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-22::DEBUG::2012-12-11 11:55:17,009::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-22::DEBUG::2012-12-11 11:55:17,010::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-22::DEBUG::2012-12-11 11:55:17,010::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-22::DEBUG::2012-12-11 11:55:17,010::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-22::DEBUG::2012-12-11 11:55:17,010::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-22::INFO::2012-12-11 11:55:17,010::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-22::DEBUG::2012-12-11 11:55:17,010::task::1172::TaskManager.Task::(prepare) Task=`c60a7cf8-8d18-468a-a069-972307c79818`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-22::DEBUG::2012-12-11 11:55:17,011::task::588::TaskManager.Task::(_updateState) Task=`c60a7cf8-8d18-468a-a069-972307c79818`::moving from state preparing -> state finished >Thread-22::DEBUG::2012-12-11 11:55:17,011::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-22::DEBUG::2012-12-11 11:55:17,011::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-22::DEBUG::2012-12-11 11:55:17,011::task::978::TaskManager.Task::(_decref) Task=`c60a7cf8-8d18-468a-a069-972307c79818`::ref 0 aborting False >Thread-23::DEBUG::2012-12-11 11:55:17,115::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-23::DEBUG::2012-12-11 11:55:17,116::task::588::TaskManager.Task::(_updateState) Task=`d852062c-bd99-4b76-8388-f57fdc4f9e75`::moving from state init -> state preparing >Thread-23::INFO::2012-12-11 11:55:17,116::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=1, storageType=1, remotePath='10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', options=None) >Thread-23::DEBUG::2012-12-11 11:55:17,116::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-23::DEBUG::2012-12-11 11:55:17,116::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-23::DEBUG::2012-12-11 11:55:17,116::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-23::DEBUG::2012-12-11 11:55:17,116::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-23::DEBUG::2012-12-11 11:55:17,117::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-23::DEBUG::2012-12-11 11:55:17,128::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-23::DEBUG::2012-12-11 11:55:17,129::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-23::DEBUG::2012-12-11 11:55:18,963::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-23::DEBUG::2012-12-11 11:55:19,004::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-23::DEBUG::2012-12-11 11:55:19,004::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2012-12-11 11:55:19,004::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2012-12-11 11:55:19,004::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2012-12-11 11:55:19,005::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2012-12-11 11:55:19,005::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-23::DEBUG::2012-12-11 11:55:19,005::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-23::DEBUG::2012-12-11 11:55:19,005::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-23::DEBUG::2012-12-11 11:55:19,005::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-23::DEBUG::2012-12-11 11:55:19,006::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-23::DEBUG::2012-12-11 11:55:19,029::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-23::DEBUG::2012-12-11 11:55:19,029::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-23::INFO::2012-12-11 11:55:19,252::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-23::DEBUG::2012-12-11 11:55:19,252::task::1172::TaskManager.Task::(prepare) Task=`d852062c-bd99-4b76-8388-f57fdc4f9e75`::finished: {'domlist': []} >Thread-23::DEBUG::2012-12-11 11:55:19,252::task::588::TaskManager.Task::(_updateState) Task=`d852062c-bd99-4b76-8388-f57fdc4f9e75`::moving from state preparing -> state finished >Thread-23::DEBUG::2012-12-11 11:55:19,252::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-23::DEBUG::2012-12-11 11:55:19,253::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-23::DEBUG::2012-12-11 11:55:19,253::task::978::TaskManager.Task::(_decref) Task=`d852062c-bd99-4b76-8388-f57fdc4f9e75`::ref 0 aborting False >Thread-27::DEBUG::2012-12-11 11:55:19,496::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-27::DEBUG::2012-12-11 11:55:19,497::task::588::TaskManager.Task::(_updateState) Task=`b260e5d5-ab66-4015-a8e7-1a8f4575d8b3`::moving from state init -> state preparing >Thread-27::INFO::2012-12-11 11:55:19,497::logUtils::37::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='4c83c7d8-2e16-4607-a09c-abd89369b0c0', domainName='nfsStorageTest_data_domain_0', typeSpecificArg='10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', domClass=1, domVersion='3', options=None) >Thread-27::DEBUG::2012-12-11 11:55:19,497::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-27::DEBUG::2012-12-11 11:55:19,498::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 4c83c7d8-2e16-4607-a09c-abd89369b0c0' (cwd None) >Thread-27::DEBUG::2012-12-11 11:55:19,515::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found\n'; <rc> = 5 >Thread-27::WARNING::2012-12-11 11:55:19,515::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found'] >Thread-27::DEBUG::2012-12-11 11:55:19,516::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-27::INFO::2012-12-11 11:55:19,519::nfsSD::64::Storage.StorageDomain::(create) sdUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0 domainName=nfsStorageTest_data_domain_0 remotePath=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873 domClass=1 >Thread-27::DEBUG::2012-12-11 11:55:19,531::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-27::DEBUG::2012-12-11 11:55:19,533::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-27::WARNING::2012-12-11 11:55:19,533::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-27::DEBUG::2012-12-11 11:55:19,533::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-27::DEBUG::2012-12-11 11:55:19,533::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-27::DEBUG::2012-12-11 11:55:19,533::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f997b851f7687cce0c85894568149a3481a0d88a'] >Thread-27::DEBUG::2012-12-11 11:55:19,548::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-27::DEBUG::2012-12-11 11:55:19,549::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-27::DEBUG::2012-12-11 11:55:19,549::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-27::DEBUG::2012-12-11 11:55:19,552::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f997b851f7687cce0c85894568149a3481a0d88a'] >Thread-27::DEBUG::2012-12-11 11:55:19,553::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-27::DEBUG::2012-12-11 11:55:19,554::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '4c83c7d8-2e16-4607-a09c-abd89369b0c0_imageNS' >Thread-27::DEBUG::2012-12-11 11:55:19,554::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '4c83c7d8-2e16-4607-a09c-abd89369b0c0_volumeNS' >Thread-27::DEBUG::2012-12-11 11:55:19,576::sd::354::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-27::INFO::2012-12-11 11:55:19,576::logUtils::39::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-27::DEBUG::2012-12-11 11:55:19,576::task::1172::TaskManager.Task::(prepare) Task=`b260e5d5-ab66-4015-a8e7-1a8f4575d8b3`::finished: None >Thread-27::DEBUG::2012-12-11 11:55:19,576::task::588::TaskManager.Task::(_updateState) Task=`b260e5d5-ab66-4015-a8e7-1a8f4575d8b3`::moving from state preparing -> state finished >Thread-27::DEBUG::2012-12-11 11:55:19,577::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-27::DEBUG::2012-12-11 11:55:19,577::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-27::DEBUG::2012-12-11 11:55:19,577::task::978::TaskManager.Task::(_decref) Task=`b260e5d5-ab66-4015-a8e7-1a8f4575d8b3`::ref 0 aborting False >Thread-29::DEBUG::2012-12-11 11:55:19,723::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-29::DEBUG::2012-12-11 11:55:19,724::task::588::TaskManager.Task::(_updateState) Task=`e1cd3a96-8643-402e-a8fa-724cf25a7d84`::moving from state init -> state preparing >Thread-29::INFO::2012-12-11 11:55:19,725::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='4c83c7d8-2e16-4607-a09c-abd89369b0c0', options=None) >Thread-29::DEBUG::2012-12-11 11:55:19,725::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0`ReqID=`2ead49f6-fae0-4911-8969-092063496864`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-29::DEBUG::2012-12-11 11:55:19,725::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' for lock type 'shared' >Thread-29::DEBUG::2012-12-11 11:55:19,725::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' is free. Now locking as 'shared' (1 active user) >Thread-29::DEBUG::2012-12-11 11:55:19,726::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0`ReqID=`2ead49f6-fae0-4911-8969-092063496864`::Granted request >Thread-29::DEBUG::2012-12-11 11:55:19,726::task::817::TaskManager.Task::(resourceAcquired) Task=`e1cd3a96-8643-402e-a8fa-724cf25a7d84`::_resourcesAcquired: Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0 (shared) >Thread-29::DEBUG::2012-12-11 11:55:19,726::task::978::TaskManager.Task::(_decref) Task=`e1cd3a96-8643-402e-a8fa-724cf25a7d84`::ref 1 aborting False >Thread-29::DEBUG::2012-12-11 11:55:19,726::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-29::DEBUG::2012-12-11 11:55:19,727::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 4c83c7d8-2e16-4607-a09c-abd89369b0c0' (cwd None) >Thread-29::DEBUG::2012-12-11 11:55:19,744::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found\n'; <rc> = 5 >Thread-29::WARNING::2012-12-11 11:55:19,744::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found'] >Thread-29::DEBUG::2012-12-11 11:55:19,745::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-29::DEBUG::2012-12-11 11:55:19,750::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-29::DEBUG::2012-12-11 11:55:19,750::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-29::DEBUG::2012-12-11 11:55:19,752::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f997b851f7687cce0c85894568149a3481a0d88a'] >Thread-29::DEBUG::2012-12-11 11:55:19,753::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-29::WARNING::2012-12-11 11:55:19,753::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 4c83c7d8-2e16-4607-a09c-abd89369b0c0_imageNS already registered >Thread-29::WARNING::2012-12-11 11:55:19,753::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 4c83c7d8-2e16-4607-a09c-abd89369b0c0_volumeNS already registered >Thread-29::INFO::2012-12-11 11:55:19,754::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '913568563200', 'disktotal': '1074600869888', 'mdafree': 0}} >Thread-29::DEBUG::2012-12-11 11:55:19,754::task::1172::TaskManager.Task::(prepare) Task=`e1cd3a96-8643-402e-a8fa-724cf25a7d84`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '913568563200', 'disktotal': '1074600869888', 'mdafree': 0}} >Thread-29::DEBUG::2012-12-11 11:55:19,754::task::588::TaskManager.Task::(_updateState) Task=`e1cd3a96-8643-402e-a8fa-724cf25a7d84`::moving from state preparing -> state finished >Thread-29::DEBUG::2012-12-11 11:55:19,754::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0': < ResourceRef 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0', isValid: 'True' obj: 'None'>} >Thread-29::DEBUG::2012-12-11 11:55:19,755::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-29::DEBUG::2012-12-11 11:55:19,755::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' >Thread-29::DEBUG::2012-12-11 11:55:19,755::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' (0 active users) >Thread-29::DEBUG::2012-12-11 11:55:19,755::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' is free, finding out if anyone is waiting for it. >Thread-29::DEBUG::2012-12-11 11:55:19,755::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0', Clearing records. >Thread-29::DEBUG::2012-12-11 11:55:19,755::task::978::TaskManager.Task::(_decref) Task=`e1cd3a96-8643-402e-a8fa-724cf25a7d84`::ref 0 aborting False >Thread-32::DEBUG::2012-12-11 11:55:20,934::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-32::DEBUG::2012-12-11 11:55:20,934::task::588::TaskManager.Task::(_updateState) Task=`66d7b9e2-12f8-4a51-9808-4f78acae87e0`::moving from state init -> state preparing >Thread-32::INFO::2012-12-11 11:55:20,935::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-32::INFO::2012-12-11 11:55:20,935::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-32::DEBUG::2012-12-11 11:55:20,935::task::1172::TaskManager.Task::(prepare) Task=`66d7b9e2-12f8-4a51-9808-4f78acae87e0`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-32::DEBUG::2012-12-11 11:55:20,935::task::588::TaskManager.Task::(_updateState) Task=`66d7b9e2-12f8-4a51-9808-4f78acae87e0`::moving from state preparing -> state finished >Thread-32::DEBUG::2012-12-11 11:55:20,935::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-32::DEBUG::2012-12-11 11:55:20,935::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-32::DEBUG::2012-12-11 11:55:20,935::task::978::TaskManager.Task::(_decref) Task=`66d7b9e2-12f8-4a51-9808-4f78acae87e0`::ref 0 aborting False >Thread-33::DEBUG::2012-12-11 11:55:21,063::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-33::DEBUG::2012-12-11 11:55:21,063::task::588::TaskManager.Task::(_updateState) Task=`d3f1a71f-ca47-410f-a18f-a243745926be`::moving from state init -> state preparing >Thread-33::INFO::2012-12-11 11:55:21,063::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-33::DEBUG::2012-12-11 11:55:21,067::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428 /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__11__520428' (cwd None) >Thread-33::DEBUG::2012-12-11 11:55:21,084::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-33::DEBUG::2012-12-11 11:55:21,084::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-33::DEBUG::2012-12-11 11:55:21,085::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-33::DEBUG::2012-12-11 11:55:21,085::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-33::DEBUG::2012-12-11 11:55:21,085::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-33::DEBUG::2012-12-11 11:55:21,085::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-33::INFO::2012-12-11 11:55:21,085::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-33::DEBUG::2012-12-11 11:55:21,086::task::1172::TaskManager.Task::(prepare) Task=`d3f1a71f-ca47-410f-a18f-a243745926be`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-33::DEBUG::2012-12-11 11:55:21,086::task::588::TaskManager.Task::(_updateState) Task=`d3f1a71f-ca47-410f-a18f-a243745926be`::moving from state preparing -> state finished >Thread-33::DEBUG::2012-12-11 11:55:21,086::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-33::DEBUG::2012-12-11 11:55:21,086::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-33::DEBUG::2012-12-11 11:55:21,086::task::978::TaskManager.Task::(_decref) Task=`d3f1a71f-ca47-410f-a18f-a243745926be`::ref 0 aborting False >Thread-34::DEBUG::2012-12-11 11:55:21,195::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-34::DEBUG::2012-12-11 11:55:21,196::task::588::TaskManager.Task::(_updateState) Task=`cd963ea1-78a7-44f2-ba16-4d377c6614c0`::moving from state init -> state preparing >Thread-34::INFO::2012-12-11 11:55:21,196::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=1, storageType=1, remotePath='10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', options=None) >Thread-34::DEBUG::2012-12-11 11:55:21,196::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-34::DEBUG::2012-12-11 11:55:21,196::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-34::DEBUG::2012-12-11 11:55:21,196::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-34::DEBUG::2012-12-11 11:55:21,196::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-34::DEBUG::2012-12-11 11:55:21,197::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-34::DEBUG::2012-12-11 11:55:21,208::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-34::DEBUG::2012-12-11 11:55:21,209::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-35::DEBUG::2012-12-11 11:55:21,809::task::588::TaskManager.Task::(_updateState) Task=`4f6c7aeb-b62c-4137-9a3e-e23116b6b59e`::moving from state init -> state preparing >Thread-35::INFO::2012-12-11 11:55:21,809::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-35::INFO::2012-12-11 11:55:21,809::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-35::DEBUG::2012-12-11 11:55:21,809::task::1172::TaskManager.Task::(prepare) Task=`4f6c7aeb-b62c-4137-9a3e-e23116b6b59e`::finished: {} >Thread-35::DEBUG::2012-12-11 11:55:21,810::task::588::TaskManager.Task::(_updateState) Task=`4f6c7aeb-b62c-4137-9a3e-e23116b6b59e`::moving from state preparing -> state finished >Thread-35::DEBUG::2012-12-11 11:55:21,810::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-35::DEBUG::2012-12-11 11:55:21,810::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-35::DEBUG::2012-12-11 11:55:21,810::task::978::TaskManager.Task::(_decref) Task=`4f6c7aeb-b62c-4137-9a3e-e23116b6b59e`::ref 0 aborting False >Thread-34::DEBUG::2012-12-11 11:55:23,044::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-34::DEBUG::2012-12-11 11:55:23,082::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-34::DEBUG::2012-12-11 11:55:23,082::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-34::DEBUG::2012-12-11 11:55:23,083::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-34::DEBUG::2012-12-11 11:55:23,083::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-34::DEBUG::2012-12-11 11:55:23,083::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-34::DEBUG::2012-12-11 11:55:23,083::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-34::DEBUG::2012-12-11 11:55:23,083::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-34::DEBUG::2012-12-11 11:55:23,084::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-34::DEBUG::2012-12-11 11:55:23,084::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-34::DEBUG::2012-12-11 11:55:23,084::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-34::DEBUG::2012-12-11 11:55:23,101::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-34::DEBUG::2012-12-11 11:55:23,101::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-34::INFO::2012-12-11 11:55:23,104::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-34::DEBUG::2012-12-11 11:55:23,105::task::1172::TaskManager.Task::(prepare) Task=`cd963ea1-78a7-44f2-ba16-4d377c6614c0`::finished: {'domlist': []} >Thread-34::DEBUG::2012-12-11 11:55:23,105::task::588::TaskManager.Task::(_updateState) Task=`cd963ea1-78a7-44f2-ba16-4d377c6614c0`::moving from state preparing -> state finished >Thread-34::DEBUG::2012-12-11 11:55:23,105::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-34::DEBUG::2012-12-11 11:55:23,105::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2012-12-11 11:55:23,105::task::978::TaskManager.Task::(_decref) Task=`cd963ea1-78a7-44f2-ba16-4d377c6614c0`::ref 0 aborting False >Thread-39::DEBUG::2012-12-11 11:55:23,273::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-39::DEBUG::2012-12-11 11:55:23,273::task::588::TaskManager.Task::(_updateState) Task=`4153d464-5ebb-4d94-8242-9a1c115b1b7d`::moving from state init -> state preparing >Thread-39::INFO::2012-12-11 11:55:23,273::logUtils::37::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', domainName='nfsStorageTest_data_domain_1', typeSpecificArg='10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', domClass=1, domVersion='3', options=None) >Thread-39::DEBUG::2012-12-11 11:55:23,273::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-39::DEBUG::2012-12-11 11:55:23,274::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' (cwd None) >Thread-39::DEBUG::2012-12-11 11:55:23,291::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f" not found\n'; <rc> = 5 >Thread-39::WARNING::2012-12-11 11:55:23,291::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f" not found'] >Thread-39::DEBUG::2012-12-11 11:55:23,291::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-39::INFO::2012-12-11 11:55:23,294::nfsSD::64::Storage.StorageDomain::(create) sdUUID=ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f domainName=nfsStorageTest_data_domain_1 remotePath=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428 domClass=1 >Thread-39::DEBUG::2012-12-11 11:55:23,307::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-39::DEBUG::2012-12-11 11:55:23,308::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-39::WARNING::2012-12-11 11:55:23,308::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-39::DEBUG::2012-12-11 11:55:23,308::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-39::DEBUG::2012-12-11 11:55:23,308::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-39::DEBUG::2012-12-11 11:55:23,308::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', 'ROLE=Regular', 'SDUUID=ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=8a4c40f57bff06295914b5a2f2768cf01edec779'] >Thread-39::DEBUG::2012-12-11 11:55:23,310::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-39::DEBUG::2012-12-11 11:55:23,311::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__11__520428/ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f >Thread-39::DEBUG::2012-12-11 11:55:23,311::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-39::DEBUG::2012-12-11 11:55:23,313::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', 'ROLE=Regular', 'SDUUID=ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=8a4c40f57bff06295914b5a2f2768cf01edec779'] >Thread-39::DEBUG::2012-12-11 11:55:23,314::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-39::DEBUG::2012-12-11 11:55:23,314::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f_imageNS' >Thread-39::DEBUG::2012-12-11 11:55:23,315::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f_volumeNS' >Thread-39::DEBUG::2012-12-11 11:55:23,337::sd::354::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-39::INFO::2012-12-11 11:55:23,337::logUtils::39::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-39::DEBUG::2012-12-11 11:55:23,337::task::1172::TaskManager.Task::(prepare) Task=`4153d464-5ebb-4d94-8242-9a1c115b1b7d`::finished: None >Thread-39::DEBUG::2012-12-11 11:55:23,337::task::588::TaskManager.Task::(_updateState) Task=`4153d464-5ebb-4d94-8242-9a1c115b1b7d`::moving from state preparing -> state finished >Thread-39::DEBUG::2012-12-11 11:55:23,337::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-39::DEBUG::2012-12-11 11:55:23,337::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-39::DEBUG::2012-12-11 11:55:23,337::task::978::TaskManager.Task::(_decref) Task=`4153d464-5ebb-4d94-8242-9a1c115b1b7d`::ref 0 aborting False >Thread-42::DEBUG::2012-12-11 11:55:23,398::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-42::DEBUG::2012-12-11 11:55:23,398::task::588::TaskManager.Task::(_updateState) Task=`fd237f09-d3a7-4e99-a3ed-fcc00c8449a3`::moving from state init -> state preparing >Thread-42::INFO::2012-12-11 11:55:23,398::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', options=None) >Thread-42::DEBUG::2012-12-11 11:55:23,398::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f`ReqID=`f07153a6-70a8-45d7-aa2a-9cdf40175f9e`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-42::DEBUG::2012-12-11 11:55:23,399::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' for lock type 'shared' >Thread-42::DEBUG::2012-12-11 11:55:23,399::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' is free. Now locking as 'shared' (1 active user) >Thread-42::DEBUG::2012-12-11 11:55:23,399::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f`ReqID=`f07153a6-70a8-45d7-aa2a-9cdf40175f9e`::Granted request >Thread-42::DEBUG::2012-12-11 11:55:23,399::task::817::TaskManager.Task::(resourceAcquired) Task=`fd237f09-d3a7-4e99-a3ed-fcc00c8449a3`::_resourcesAcquired: Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f (shared) >Thread-42::DEBUG::2012-12-11 11:55:23,399::task::978::TaskManager.Task::(_decref) Task=`fd237f09-d3a7-4e99-a3ed-fcc00c8449a3`::ref 1 aborting False >Thread-42::DEBUG::2012-12-11 11:55:23,400::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-42::DEBUG::2012-12-11 11:55:23,400::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' (cwd None) >Thread-42::DEBUG::2012-12-11 11:55:23,417::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f" not found\n'; <rc> = 5 >Thread-42::WARNING::2012-12-11 11:55:23,417::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f" not found'] >Thread-42::DEBUG::2012-12-11 11:55:23,418::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-42::DEBUG::2012-12-11 11:55:23,424::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__11__520428/ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f >Thread-42::DEBUG::2012-12-11 11:55:23,424::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-42::DEBUG::2012-12-11 11:55:23,426::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', 'ROLE=Regular', 'SDUUID=ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=8a4c40f57bff06295914b5a2f2768cf01edec779'] >Thread-42::DEBUG::2012-12-11 11:55:23,427::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-42::WARNING::2012-12-11 11:55:23,427::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f_imageNS already registered >Thread-42::WARNING::2012-12-11 11:55:23,427::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f_volumeNS already registered >Thread-42::INFO::2012-12-11 11:55:23,428::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '913475829760', 'disktotal': '1074600869888', 'mdafree': 0}} >Thread-42::DEBUG::2012-12-11 11:55:23,428::task::1172::TaskManager.Task::(prepare) Task=`fd237f09-d3a7-4e99-a3ed-fcc00c8449a3`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '913475829760', 'disktotal': '1074600869888', 'mdafree': 0}} >Thread-42::DEBUG::2012-12-11 11:55:23,428::task::588::TaskManager.Task::(_updateState) Task=`fd237f09-d3a7-4e99-a3ed-fcc00c8449a3`::moving from state preparing -> state finished >Thread-42::DEBUG::2012-12-11 11:55:23,428::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f': < ResourceRef 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', isValid: 'True' obj: 'None'>} >Thread-42::DEBUG::2012-12-11 11:55:23,428::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-42::DEBUG::2012-12-11 11:55:23,429::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' >Thread-42::DEBUG::2012-12-11 11:55:23,429::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' (0 active users) >Thread-42::DEBUG::2012-12-11 11:55:23,429::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' is free, finding out if anyone is waiting for it. >Thread-42::DEBUG::2012-12-11 11:55:23,429::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', Clearing records. >Thread-42::DEBUG::2012-12-11 11:55:23,429::task::978::TaskManager.Task::(_decref) Task=`fd237f09-d3a7-4e99-a3ed-fcc00c8449a3`::ref 0 aborting False >Thread-46::DEBUG::2012-12-11 11:55:24,721::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-46::DEBUG::2012-12-11 11:55:24,721::task::588::TaskManager.Task::(_updateState) Task=`22346e3f-c707-418c-a5e4-025f6b183364`::moving from state init -> state preparing >Thread-46::INFO::2012-12-11 11:55:24,721::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-46::INFO::2012-12-11 11:55:24,721::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-46::DEBUG::2012-12-11 11:55:24,721::task::1172::TaskManager.Task::(prepare) Task=`22346e3f-c707-418c-a5e4-025f6b183364`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-46::DEBUG::2012-12-11 11:55:24,722::task::588::TaskManager.Task::(_updateState) Task=`22346e3f-c707-418c-a5e4-025f6b183364`::moving from state preparing -> state finished >Thread-46::DEBUG::2012-12-11 11:55:24,722::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-46::DEBUG::2012-12-11 11:55:24,722::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-46::DEBUG::2012-12-11 11:55:24,722::task::978::TaskManager.Task::(_decref) Task=`22346e3f-c707-418c-a5e4-025f6b183364`::ref 0 aborting False >Thread-47::DEBUG::2012-12-11 11:55:24,777::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-47::DEBUG::2012-12-11 11:55:24,777::task::588::TaskManager.Task::(_updateState) Task=`4103dd8f-6205-4e18-a703-424f2b76b228`::moving from state init -> state preparing >Thread-47::INFO::2012-12-11 11:55:24,777::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': ''}], options=None) >Thread-47::DEBUG::2012-12-11 11:55:24,781::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6 10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558 /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121211115212316558' (cwd None) >Thread-47::DEBUG::2012-12-11 11:55:24,941::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-47::DEBUG::2012-12-11 11:55:24,941::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-47::DEBUG::2012-12-11 11:55:24,941::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-47::DEBUG::2012-12-11 11:55:24,941::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-47::DEBUG::2012-12-11 11:55:24,942::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-47::DEBUG::2012-12-11 11:55:24,942::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-47::INFO::2012-12-11 11:55:24,942::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-47::DEBUG::2012-12-11 11:55:24,942::task::1172::TaskManager.Task::(prepare) Task=`4103dd8f-6205-4e18-a703-424f2b76b228`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-47::DEBUG::2012-12-11 11:55:24,942::task::588::TaskManager.Task::(_updateState) Task=`4103dd8f-6205-4e18-a703-424f2b76b228`::moving from state preparing -> state finished >Thread-47::DEBUG::2012-12-11 11:55:24,942::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-47::DEBUG::2012-12-11 11:55:24,943::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-47::DEBUG::2012-12-11 11:55:24,943::task::978::TaskManager.Task::(_decref) Task=`4103dd8f-6205-4e18-a703-424f2b76b228`::ref 0 aborting False >Thread-48::DEBUG::2012-12-11 11:55:25,004::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-48::DEBUG::2012-12-11 11:55:25,005::task::588::TaskManager.Task::(_updateState) Task=`5cfb3748-362b-44b5-b35a-b670855cce00`::moving from state init -> state preparing >Thread-48::INFO::2012-12-11 11:55:25,005::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=2, storageType=1, remotePath='10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558', options=None) >Thread-48::DEBUG::2012-12-11 11:55:25,005::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-48::DEBUG::2012-12-11 11:55:25,005::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-48::DEBUG::2012-12-11 11:55:25,005::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-48::DEBUG::2012-12-11 11:55:25,005::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-48::DEBUG::2012-12-11 11:55:25,006::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-48::DEBUG::2012-12-11 11:55:25,018::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-48::DEBUG::2012-12-11 11:55:25,019::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-48::DEBUG::2012-12-11 11:55:26,853::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-48::DEBUG::2012-12-11 11:55:26,888::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-48::DEBUG::2012-12-11 11:55:26,889::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-48::DEBUG::2012-12-11 11:55:26,889::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-48::DEBUG::2012-12-11 11:55:26,889::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-48::DEBUG::2012-12-11 11:55:26,889::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-48::DEBUG::2012-12-11 11:55:26,890::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-48::DEBUG::2012-12-11 11:55:26,890::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-48::DEBUG::2012-12-11 11:55:26,890::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-48::DEBUG::2012-12-11 11:55:26,890::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-48::DEBUG::2012-12-11 11:55:26,891::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-48::DEBUG::2012-12-11 11:55:26,907::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-48::DEBUG::2012-12-11 11:55:26,907::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-48::INFO::2012-12-11 11:55:26,910::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-48::DEBUG::2012-12-11 11:55:26,911::task::1172::TaskManager.Task::(prepare) Task=`5cfb3748-362b-44b5-b35a-b670855cce00`::finished: {'domlist': []} >Thread-48::DEBUG::2012-12-11 11:55:26,911::task::588::TaskManager.Task::(_updateState) Task=`5cfb3748-362b-44b5-b35a-b670855cce00`::moving from state preparing -> state finished >Thread-48::DEBUG::2012-12-11 11:55:26,911::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-48::DEBUG::2012-12-11 11:55:26,911::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-48::DEBUG::2012-12-11 11:55:26,911::task::978::TaskManager.Task::(_decref) Task=`5cfb3748-362b-44b5-b35a-b670855cce00`::ref 0 aborting False >Thread-53::DEBUG::2012-12-11 11:55:27,079::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-53::DEBUG::2012-12-11 11:55:27,079::task::588::TaskManager.Task::(_updateState) Task=`f4bd8524-9070-4ec6-baea-cfc4313bd8c9`::moving from state init -> state preparing >Thread-53::INFO::2012-12-11 11:55:27,079::logUtils::37::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='30067796-d127-493d-8dae-e9e81613e08c', domainName='nfsStorageTest_ISO', typeSpecificArg='10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558', domClass=2, domVersion='0', options=None) >Thread-53::DEBUG::2012-12-11 11:55:27,079::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-53::DEBUG::2012-12-11 11:55:27,080::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 30067796-d127-493d-8dae-e9e81613e08c' (cwd None) >Thread-53::DEBUG::2012-12-11 11:55:27,097::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "30067796-d127-493d-8dae-e9e81613e08c" not found\n'; <rc> = 5 >Thread-53::WARNING::2012-12-11 11:55:27,097::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "30067796-d127-493d-8dae-e9e81613e08c" not found'] >Thread-53::DEBUG::2012-12-11 11:55:27,097::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-53::INFO::2012-12-11 11:55:27,100::nfsSD::64::Storage.StorageDomain::(create) sdUUID=30067796-d127-493d-8dae-e9e81613e08c domainName=nfsStorageTest_ISO remotePath=10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558 domClass=2 >Thread-53::DEBUG::2012-12-11 11:55:27,169::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-53::DEBUG::2012-12-11 11:55:27,171::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-53::WARNING::2012-12-11 11:55:27,171::persistentDict::248::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-53::DEBUG::2012-12-11 11:55:27,171::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-53::DEBUG::2012-12-11 11:55:27,171::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-53::DEBUG::2012-12-11 11:55:27,171::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=nfsStorageTest_ISO', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558', 'ROLE=Regular', 'SDUUID=30067796-d127-493d-8dae-e9e81613e08c', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=1a97774c10967745a7e722106545a32da9b80f24'] >Thread-53::DEBUG::2012-12-11 11:55:27,173::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-53::DEBUG::2012-12-11 11:55:27,176::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121211115212316558/30067796-d127-493d-8dae-e9e81613e08c >Thread-53::DEBUG::2012-12-11 11:55:27,176::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-53::DEBUG::2012-12-11 11:55:27,179::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=nfsStorageTest_ISO', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558', 'ROLE=Regular', 'SDUUID=30067796-d127-493d-8dae-e9e81613e08c', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=1a97774c10967745a7e722106545a32da9b80f24'] >Thread-53::DEBUG::2012-12-11 11:55:27,181::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-53::DEBUG::2012-12-11 11:55:27,181::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '30067796-d127-493d-8dae-e9e81613e08c_imageNS' >Thread-53::DEBUG::2012-12-11 11:55:27,182::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace '30067796-d127-493d-8dae-e9e81613e08c_volumeNS' >Thread-53::DEBUG::2012-12-11 11:55:27,182::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/libexec/vdsm/safelease release -f /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121211115212316558/30067796-d127-493d-8dae-e9e81613e08c/dom_md/leases 0' (cwd /usr/libexec/vdsm) >Thread-53::DEBUG::2012-12-11 11:55:27,202::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-53::DEBUG::2012-12-11 11:55:27,202::sd::354::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-53::INFO::2012-12-11 11:55:27,202::logUtils::39::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-53::DEBUG::2012-12-11 11:55:27,202::task::1172::TaskManager.Task::(prepare) Task=`f4bd8524-9070-4ec6-baea-cfc4313bd8c9`::finished: None >Thread-53::DEBUG::2012-12-11 11:55:27,202::task::588::TaskManager.Task::(_updateState) Task=`f4bd8524-9070-4ec6-baea-cfc4313bd8c9`::moving from state preparing -> state finished >Thread-53::DEBUG::2012-12-11 11:55:27,203::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-53::DEBUG::2012-12-11 11:55:27,203::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-53::DEBUG::2012-12-11 11:55:27,203::task::978::TaskManager.Task::(_decref) Task=`f4bd8524-9070-4ec6-baea-cfc4313bd8c9`::ref 0 aborting False >Thread-57::DEBUG::2012-12-11 11:55:27,262::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-57::DEBUG::2012-12-11 11:55:27,262::task::588::TaskManager.Task::(_updateState) Task=`10d28423-856f-46a1-b6f8-8be32e644ac8`::moving from state init -> state preparing >Thread-57::INFO::2012-12-11 11:55:27,262::logUtils::37::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='30067796-d127-493d-8dae-e9e81613e08c', options=None) >Thread-57::DEBUG::2012-12-11 11:55:27,262::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.30067796-d127-493d-8dae-e9e81613e08c`ReqID=`2bd440fe-6132-4494-afe9-4036ced05205`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-57::DEBUG::2012-12-11 11:55:27,263::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.30067796-d127-493d-8dae-e9e81613e08c' for lock type 'shared' >Thread-57::DEBUG::2012-12-11 11:55:27,263::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.30067796-d127-493d-8dae-e9e81613e08c' is free. Now locking as 'shared' (1 active user) >Thread-57::DEBUG::2012-12-11 11:55:27,263::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.30067796-d127-493d-8dae-e9e81613e08c`ReqID=`2bd440fe-6132-4494-afe9-4036ced05205`::Granted request >Thread-57::DEBUG::2012-12-11 11:55:27,263::task::817::TaskManager.Task::(resourceAcquired) Task=`10d28423-856f-46a1-b6f8-8be32e644ac8`::_resourcesAcquired: Storage.30067796-d127-493d-8dae-e9e81613e08c (shared) >Thread-57::DEBUG::2012-12-11 11:55:27,263::task::978::TaskManager.Task::(_decref) Task=`10d28423-856f-46a1-b6f8-8be32e644ac8`::ref 1 aborting False >Thread-57::DEBUG::2012-12-11 11:55:27,264::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2012-12-11 11:55:27,264::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 30067796-d127-493d-8dae-e9e81613e08c' (cwd None) >Thread-57::DEBUG::2012-12-11 11:55:27,281::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "30067796-d127-493d-8dae-e9e81613e08c" not found\n'; <rc> = 5 >Thread-57::WARNING::2012-12-11 11:55:27,281::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "30067796-d127-493d-8dae-e9e81613e08c" not found'] >Thread-57::DEBUG::2012-12-11 11:55:27,282::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2012-12-11 11:55:27,288::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121211115212316558/30067796-d127-493d-8dae-e9e81613e08c >Thread-57::DEBUG::2012-12-11 11:55:27,288::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-57::DEBUG::2012-12-11 11:55:27,290::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=nfsStorageTest_ISO', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.16.27:/volumes/shafan/vm-18-155_nfs_20121211115212316558', 'ROLE=Regular', 'SDUUID=30067796-d127-493d-8dae-e9e81613e08c', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=1a97774c10967745a7e722106545a32da9b80f24'] >Thread-57::DEBUG::2012-12-11 11:55:27,291::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-57::WARNING::2012-12-11 11:55:27,292::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 30067796-d127-493d-8dae-e9e81613e08c_imageNS already registered >Thread-57::WARNING::2012-12-11 11:55:27,292::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 30067796-d127-493d-8dae-e9e81613e08c_volumeNS already registered >Thread-57::INFO::2012-12-11 11:55:27,293::logUtils::39::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '842754162688', 'disktotal': '842754162688', 'mdafree': 0}} >Thread-57::DEBUG::2012-12-11 11:55:27,293::task::1172::TaskManager.Task::(prepare) Task=`10d28423-856f-46a1-b6f8-8be32e644ac8`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '842754162688', 'disktotal': '842754162688', 'mdafree': 0}} >Thread-57::DEBUG::2012-12-11 11:55:27,293::task::588::TaskManager.Task::(_updateState) Task=`10d28423-856f-46a1-b6f8-8be32e644ac8`::moving from state preparing -> state finished >Thread-57::DEBUG::2012-12-11 11:55:27,293::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.30067796-d127-493d-8dae-e9e81613e08c': < ResourceRef 'Storage.30067796-d127-493d-8dae-e9e81613e08c', isValid: 'True' obj: 'None'>} >Thread-57::DEBUG::2012-12-11 11:55:27,293::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-57::DEBUG::2012-12-11 11:55:27,294::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.30067796-d127-493d-8dae-e9e81613e08c' >Thread-57::DEBUG::2012-12-11 11:55:27,294::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.30067796-d127-493d-8dae-e9e81613e08c' (0 active users) >Thread-57::DEBUG::2012-12-11 11:55:27,294::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.30067796-d127-493d-8dae-e9e81613e08c' is free, finding out if anyone is waiting for it. >Thread-57::DEBUG::2012-12-11 11:55:27,294::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.30067796-d127-493d-8dae-e9e81613e08c', Clearing records. >Thread-57::DEBUG::2012-12-11 11:55:27,294::task::978::TaskManager.Task::(_decref) Task=`10d28423-856f-46a1-b6f8-8be32e644ac8`::ref 0 aborting False >Thread-62::DEBUG::2012-12-11 11:55:28,513::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-62::DEBUG::2012-12-11 11:55:28,513::task::588::TaskManager.Task::(_updateState) Task=`24653164-0f08-405a-83fa-9091f26da0e0`::moving from state init -> state preparing >Thread-62::INFO::2012-12-11 11:55:28,513::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5', 'port': ''}], options=None) >Thread-62::INFO::2012-12-11 11:55:28,513::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-62::DEBUG::2012-12-11 11:55:28,513::task::1172::TaskManager.Task::(prepare) Task=`24653164-0f08-405a-83fa-9091f26da0e0`::finished: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-62::DEBUG::2012-12-11 11:55:28,514::task::588::TaskManager.Task::(_updateState) Task=`24653164-0f08-405a-83fa-9091f26da0e0`::moving from state preparing -> state finished >Thread-62::DEBUG::2012-12-11 11:55:28,514::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-62::DEBUG::2012-12-11 11:55:28,514::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-62::DEBUG::2012-12-11 11:55:28,514::task::978::TaskManager.Task::(_decref) Task=`24653164-0f08-405a-83fa-9091f26da0e0`::ref 0 aborting False >Thread-63::DEBUG::2012-12-11 11:55:28,577::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-63::DEBUG::2012-12-11 11:55:28,577::task::588::TaskManager.Task::(_updateState) Task=`55d734c6-ecf1-48aa-8ba8-ae18f3dbe33f`::moving from state init -> state preparing >Thread-63::INFO::2012-12-11 11:55:28,577::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5', 'port': ''}], options=None) >Thread-63::DEBUG::2012-12-11 11:55:28,581::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-63::DEBUG::2012-12-11 11:55:28,581::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-63::DEBUG::2012-12-11 11:55:28,581::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-63::DEBUG::2012-12-11 11:55:28,581::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-63::DEBUG::2012-12-11 11:55:28,581::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-63::DEBUG::2012-12-11 11:55:28,582::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-63::INFO::2012-12-11 11:55:28,582::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-63::DEBUG::2012-12-11 11:55:28,582::task::1172::TaskManager.Task::(prepare) Task=`55d734c6-ecf1-48aa-8ba8-ae18f3dbe33f`::finished: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-63::DEBUG::2012-12-11 11:55:28,582::task::588::TaskManager.Task::(_updateState) Task=`55d734c6-ecf1-48aa-8ba8-ae18f3dbe33f`::moving from state preparing -> state finished >Thread-63::DEBUG::2012-12-11 11:55:28,582::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-63::DEBUG::2012-12-11 11:55:28,582::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-63::DEBUG::2012-12-11 11:55:28,582::task::978::TaskManager.Task::(_decref) Task=`55d734c6-ecf1-48aa-8ba8-ae18f3dbe33f`::ref 0 aborting False >Thread-64::DEBUG::2012-12-11 11:55:28,644::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-64::DEBUG::2012-12-11 11:55:28,645::task::588::TaskManager.Task::(_updateState) Task=`5118a064-093b-4fc1-884f-4579a0a46ca7`::moving from state init -> state preparing >Thread-64::INFO::2012-12-11 11:55:28,645::logUtils::37::dispatcher::(wrapper) Run and protect: createStoragePool(poolType=None, spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', poolName='nfsStorageTest', masterDom='4c83c7d8-2e16-4607-a09c-abd89369b0c0', domList=['4c83c7d8-2e16-4607-a09c-abd89369b0c0'], masterVersion=1, lockPolicy=None, lockRenewalIntervalSec=5, leaseTimeSec=60, ioOpTimeoutSec=10, leaseRetries=3, options=None) >Thread-64::DEBUG::2012-12-11 11:55:28,645::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-64::DEBUG::2012-12-11 11:55:28,645::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-64::DEBUG::2012-12-11 11:55:28,645::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-64::DEBUG::2012-12-11 11:55:28,645::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-64::DEBUG::2012-12-11 11:55:28,646::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-64::DEBUG::2012-12-11 11:55:28,657::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-64::DEBUG::2012-12-11 11:55:28,658::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-64::DEBUG::2012-12-11 11:55:30,493::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-64::DEBUG::2012-12-11 11:55:30,530::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-64::DEBUG::2012-12-11 11:55:30,531::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:30,531::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:30,531::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:30,531::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:30,531::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:30,532::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:30,532::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-64::DEBUG::2012-12-11 11:55:30,532::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:30,532::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 4c83c7d8-2e16-4607-a09c-abd89369b0c0' (cwd None) >Thread-64::DEBUG::2012-12-11 11:55:30,549::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found\n'; <rc> = 5 >Thread-64::WARNING::2012-12-11 11:55:30,550::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found'] >Thread-64::DEBUG::2012-12-11 11:55:30,550::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:30,556::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-64::DEBUG::2012-12-11 11:55:30,556::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-64::DEBUG::2012-12-11 11:55:30,558::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f997b851f7687cce0c85894568149a3481a0d88a'] >Thread-64::DEBUG::2012-12-11 11:55:30,559::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-64::WARNING::2012-12-11 11:55:30,559::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 4c83c7d8-2e16-4607-a09c-abd89369b0c0_imageNS already registered >Thread-64::WARNING::2012-12-11 11:55:30,559::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 4c83c7d8-2e16-4607-a09c-abd89369b0c0_volumeNS already registered >Thread-64::INFO::2012-12-11 11:55:30,560::fileSD::260::Storage.StorageDomain::(validate) sdUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-64::DEBUG::2012-12-11 11:55:30,561::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f997b851f7687cce0c85894568149a3481a0d88a'] >Thread-64::DEBUG::2012-12-11 11:55:30,562::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`82fc473a-91dd-4ced-836c-9241b78383c3`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-64::DEBUG::2012-12-11 11:55:30,562::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'exclusive' >Thread-64::DEBUG::2012-12-11 11:55:30,562::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'exclusive' (1 active user) >Thread-64::DEBUG::2012-12-11 11:55:30,562::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`82fc473a-91dd-4ced-836c-9241b78383c3`::Granted request >Thread-64::DEBUG::2012-12-11 11:55:30,563::task::817::TaskManager.Task::(resourceAcquired) Task=`5118a064-093b-4fc1-884f-4579a0a46ca7`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (exclusive) >Thread-64::DEBUG::2012-12-11 11:55:30,563::task::978::TaskManager.Task::(_decref) Task=`5118a064-093b-4fc1-884f-4579a0a46ca7`::ref 1 aborting False >Thread-64::DEBUG::2012-12-11 11:55:30,563::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0`ReqID=`b03736d3-f04c-4468-8e58-a9648e528ba1`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-64::DEBUG::2012-12-11 11:55:30,563::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' for lock type 'exclusive' >Thread-64::DEBUG::2012-12-11 11:55:30,563::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' is free. Now locking as 'exclusive' (1 active user) >Thread-64::DEBUG::2012-12-11 11:55:30,563::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0`ReqID=`b03736d3-f04c-4468-8e58-a9648e528ba1`::Granted request >Thread-64::DEBUG::2012-12-11 11:55:30,564::task::817::TaskManager.Task::(resourceAcquired) Task=`5118a064-093b-4fc1-884f-4579a0a46ca7`::_resourcesAcquired: Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0 (exclusive) >Thread-64::DEBUG::2012-12-11 11:55:30,564::task::978::TaskManager.Task::(_decref) Task=`5118a064-093b-4fc1-884f-4579a0a46ca7`::ref 1 aborting False >Thread-64::INFO::2012-12-11 11:55:30,564::sp::542::Storage.StoragePool::(create) spUUID=d32a8696-910b-4222-a364-1c00b5303fbb poolName=nfsStorageTest master_sd=4c83c7d8-2e16-4607-a09c-abd89369b0c0 domList=['4c83c7d8-2e16-4607-a09c-abd89369b0c0'] masterVersion=1 {'LEASETIMESEC': 60, 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3, 'LOCKRENEWALINTERVALSEC': 5} >Thread-64::INFO::2012-12-11 11:55:30,564::fileSD::260::Storage.StorageDomain::(validate) sdUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-64::DEBUG::2012-12-11 11:55:30,566::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f997b851f7687cce0c85894568149a3481a0d88a'] >Thread-64::DEBUG::2012-12-11 11:55:30,567::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f997b851f7687cce0c85894568149a3481a0d88a'] >Thread-64::DEBUG::2012-12-11 11:55:30,568::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-64::DEBUG::2012-12-11 11:55:30,568::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-64::DEBUG::2012-12-11 11:55:30,568::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=ce4206b343bf1450b3f60305e0e94b28c9cbd1d0'] >Thread-64::DEBUG::2012-12-11 11:55:30,570::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-64::INFO::2012-12-11 11:55:30,570::safelease::156::SANLock::(acquireHostId) Acquiring host id for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 (id: 250) >Thread-69::DEBUG::2012-12-11 11:55:32,424::task::588::TaskManager.Task::(_updateState) Task=`6d79d2d1-1914-49ef-a3d2-c24c9a48deb4`::moving from state init -> state preparing >Thread-69::INFO::2012-12-11 11:55:32,424::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-69::INFO::2012-12-11 11:55:32,424::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-69::DEBUG::2012-12-11 11:55:32,424::task::1172::TaskManager.Task::(prepare) Task=`6d79d2d1-1914-49ef-a3d2-c24c9a48deb4`::finished: {} >Thread-69::DEBUG::2012-12-11 11:55:32,424::task::588::TaskManager.Task::(_updateState) Task=`6d79d2d1-1914-49ef-a3d2-c24c9a48deb4`::moving from state preparing -> state finished >Thread-69::DEBUG::2012-12-11 11:55:32,424::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-69::DEBUG::2012-12-11 11:55:32,425::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-69::DEBUG::2012-12-11 11:55:32,425::task::978::TaskManager.Task::(_decref) Task=`6d79d2d1-1914-49ef-a3d2-c24c9a48deb4`::ref 0 aborting False >Thread-75::DEBUG::2012-12-11 11:55:42,820::task::588::TaskManager.Task::(_updateState) Task=`662a1ee5-6af7-4472-9c28-993451cf76bd`::moving from state init -> state preparing >Thread-75::INFO::2012-12-11 11:55:42,820::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-75::INFO::2012-12-11 11:55:42,820::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-75::DEBUG::2012-12-11 11:55:42,821::task::1172::TaskManager.Task::(prepare) Task=`662a1ee5-6af7-4472-9c28-993451cf76bd`::finished: {} >Thread-75::DEBUG::2012-12-11 11:55:42,821::task::588::TaskManager.Task::(_updateState) Task=`662a1ee5-6af7-4472-9c28-993451cf76bd`::moving from state preparing -> state finished >Thread-75::DEBUG::2012-12-11 11:55:42,821::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-75::DEBUG::2012-12-11 11:55:42,821::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-75::DEBUG::2012-12-11 11:55:42,821::task::978::TaskManager.Task::(_decref) Task=`662a1ee5-6af7-4472-9c28-993451cf76bd`::ref 0 aborting False >Thread-64::DEBUG::2012-12-11 11:55:51,574::safelease::166::SANLock::(acquireHostId) Host id for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 successfully acquired (id: 250) >Thread-64::INFO::2012-12-11 11:55:51,574::safelease::188::SANLock::(acquire) Acquiring cluster lock for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 (id: 250) >Thread-64::DEBUG::2012-12-11 11:55:51,615::safelease::213::SANLock::(acquire) Cluster lock for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 successfully acquired (id: 250) >Thread-64::INFO::2012-12-11 11:55:51,615::sp::721::Storage.StoragePool::(createMaster) setting master domain for spUUID d32a8696-910b-4222-a364-1c00b5303fbb on sdUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-64::DEBUG::2012-12-11 11:55:51,615::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >Thread-64::DEBUG::2012-12-11 11:55:51,615::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-64::DEBUG::2012-12-11 11:55:51,618::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Regular', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=ce4206b343bf1450b3f60305e0e94b28c9cbd1d0'] >Thread-64::DEBUG::2012-12-11 11:55:51,618::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-64::DEBUG::2012-12-11 11:55:51,619::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-64::DEBUG::2012-12-11 11:55:51,619::persistentDict::155::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-64::DEBUG::2012-12-11 11:55:51,619::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >Thread-64::DEBUG::2012-12-11 11:55:51,619::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=d32a8696-910b-4222-a364-1c00b5303fbb', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Master', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dfd43e5b9b700aadd22d3a8e46f80d8998686477'] >Thread-64::DEBUG::2012-12-11 11:55:51,621::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >Thread-64::DEBUG::2012-12-11 11:55:51,621::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 verified, version 1 >Thread-64::DEBUG::2012-12-11 11:55:51,621::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-64::DEBUG::2012-12-11 11:55:51,621::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-64::DEBUG::2012-12-11 11:55:51,623::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=d32a8696-910b-4222-a364-1c00b5303fbb', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Master', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dfd43e5b9b700aadd22d3a8e46f80d8998686477'] >Thread-64::DEBUG::2012-12-11 11:55:51,624::sp::1495::Storage.StoragePool::(updateMonitoringThreads) sp `d32a8696-910b-4222-a364-1c00b5303fbb` started monitoring domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` >Thread-64::DEBUG::2012-12-11 11:55:51,625::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-64::DEBUG::2012-12-11 11:55:51,626::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-64::DEBUG::2012-12-11 11:55:51,627::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-64::DEBUG::2012-12-11 11:55:51,647::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-64::DEBUG::2012-12-11 11:55:51,647::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-64::INFO::2012-12-11 11:55:51,650::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0 to /rhev/data-center/d32a8696-910b-4222-a364-1c00b5303fbb/4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-64::INFO::2012-12-11 11:55:51,650::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0 to /rhev/data-center/d32a8696-910b-4222-a364-1c00b5303fbb/mastersd >Thread-64::INFO::2012-12-11 11:55:51,651::safelease::218::SANLock::(release) Releasing cluster lock for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 (id: 250) >Thread-64::DEBUG::2012-12-11 11:55:52,356::safelease::229::SANLock::(release) Cluster lock for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 successfully released (id: 250) >Thread-64::INFO::2012-12-11 11:55:52,356::safelease::171::SANLock::(releaseHostId) Releasing host id for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 (id: 250) >Thread-85::DEBUG::2012-12-11 11:55:53,202::task::588::TaskManager.Task::(_updateState) Task=`d3f08244-4929-4ce2-aabb-ba8a80c4f406`::moving from state init -> state preparing >Thread-85::INFO::2012-12-11 11:55:53,202::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-85::INFO::2012-12-11 11:55:53,202::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-85::DEBUG::2012-12-11 11:55:53,202::task::1172::TaskManager.Task::(prepare) Task=`d3f08244-4929-4ce2-aabb-ba8a80c4f406`::finished: {} >Thread-85::DEBUG::2012-12-11 11:55:53,202::task::588::TaskManager.Task::(_updateState) Task=`d3f08244-4929-4ce2-aabb-ba8a80c4f406`::moving from state preparing -> state finished >Thread-85::DEBUG::2012-12-11 11:55:53,202::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2012-12-11 11:55:53,203::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2012-12-11 11:55:53,203::task::978::TaskManager.Task::(_decref) Task=`d3f08244-4929-4ce2-aabb-ba8a80c4f406`::ref 0 aborting False >Thread-64::DEBUG::2012-12-11 11:55:55,357::safelease::180::SANLock::(releaseHostId) Host id for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 released successfully (id: None) >Thread-64::INFO::2012-12-11 11:55:55,357::logUtils::39::dispatcher::(wrapper) Run and protect: createStoragePool, Return response: True >Thread-64::DEBUG::2012-12-11 11:55:55,358::task::1172::TaskManager.Task::(prepare) Task=`5118a064-093b-4fc1-884f-4579a0a46ca7`::finished: True >Thread-64::DEBUG::2012-12-11 11:55:55,358::task::588::TaskManager.Task::(_updateState) Task=`5118a064-093b-4fc1-884f-4579a0a46ca7`::moving from state preparing -> state finished >Thread-64::DEBUG::2012-12-11 11:55:55,358::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>, 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0': < ResourceRef 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0', isValid: 'True' obj: 'None'>} >Thread-64::DEBUG::2012-12-11 11:55:55,358::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-64::DEBUG::2012-12-11 11:55:55,359::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-64::DEBUG::2012-12-11 11:55:55,359::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-64::DEBUG::2012-12-11 11:55:55,359::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-64::DEBUG::2012-12-11 11:55:55,359::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-64::DEBUG::2012-12-11 11:55:55,359::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' >Thread-64::DEBUG::2012-12-11 11:55:55,359::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' (0 active users) >Thread-64::DEBUG::2012-12-11 11:55:55,360::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' is free, finding out if anyone is waiting for it. >Thread-64::DEBUG::2012-12-11 11:55:55,360::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0', Clearing records. >Thread-64::DEBUG::2012-12-11 11:55:55,360::task::978::TaskManager.Task::(_decref) Task=`5118a064-093b-4fc1-884f-4579a0a46ca7`::ref 0 aborting False >Thread-88::DEBUG::2012-12-11 11:55:55,519::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-88::DEBUG::2012-12-11 11:55:55,519::task::588::TaskManager.Task::(_updateState) Task=`3d7d9ef5-09fd-4254-be20-57e129c992fc`::moving from state init -> state preparing >Thread-88::INFO::2012-12-11 11:55:55,519::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5', 'port': ''}], options=None) >Thread-88::INFO::2012-12-11 11:55:55,519::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-88::DEBUG::2012-12-11 11:55:55,520::task::1172::TaskManager.Task::(prepare) Task=`3d7d9ef5-09fd-4254-be20-57e129c992fc`::finished: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-88::DEBUG::2012-12-11 11:55:55,520::task::588::TaskManager.Task::(_updateState) Task=`3d7d9ef5-09fd-4254-be20-57e129c992fc`::moving from state preparing -> state finished >Thread-88::DEBUG::2012-12-11 11:55:55,520::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-88::DEBUG::2012-12-11 11:55:55,520::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-88::DEBUG::2012-12-11 11:55:55,520::task::978::TaskManager.Task::(_decref) Task=`3d7d9ef5-09fd-4254-be20-57e129c992fc`::ref 0 aborting False >Thread-89::DEBUG::2012-12-11 11:55:55,609::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-89::DEBUG::2012-12-11 11:55:55,610::task::588::TaskManager.Task::(_updateState) Task=`4e07ad4a-cab8-4029-af04-1ea4d31697c8`::moving from state init -> state preparing >Thread-89::INFO::2012-12-11 11:55:55,610::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5', 'port': ''}], options=None) >Thread-89::DEBUG::2012-12-11 11:55:55,614::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-89::DEBUG::2012-12-11 11:55:55,614::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-89::DEBUG::2012-12-11 11:55:55,614::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-89::DEBUG::2012-12-11 11:55:55,614::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-89::DEBUG::2012-12-11 11:55:55,614::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-89::DEBUG::2012-12-11 11:55:55,615::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-89::INFO::2012-12-11 11:55:55,615::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-89::DEBUG::2012-12-11 11:55:55,615::task::1172::TaskManager.Task::(prepare) Task=`4e07ad4a-cab8-4029-af04-1ea4d31697c8`::finished: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-89::DEBUG::2012-12-11 11:55:55,615::task::588::TaskManager.Task::(_updateState) Task=`4e07ad4a-cab8-4029-af04-1ea4d31697c8`::moving from state preparing -> state finished >Thread-89::DEBUG::2012-12-11 11:55:55,615::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2012-12-11 11:55:55,615::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2012-12-11 11:55:55,615::task::978::TaskManager.Task::(_decref) Task=`4e07ad4a-cab8-4029-af04-1ea4d31697c8`::ref 0 aborting False >Thread-90::DEBUG::2012-12-11 11:55:55,721::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-90::DEBUG::2012-12-11 11:55:55,721::task::588::TaskManager.Task::(_updateState) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::moving from state init -> state preparing >Thread-90::INFO::2012-12-11 11:55:55,721::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-90::ERROR::2012-12-11 11:55:55,722::task::853::TaskManager.Task::(_setError) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 861, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 38, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 528, in getSpmStatus > pool = self.getPool(spUUID) > File "/usr/share/vdsm/storage/hsm.py", line 265, in getPool > raise se.StoragePoolUnknown(spUUID) >StoragePoolUnknown: Unknown pool id, pool not connected: ('d32a8696-910b-4222-a364-1c00b5303fbb',) >Thread-90::DEBUG::2012-12-11 11:55:55,723::task::872::TaskManager.Task::(_run) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::Task._run: 59477efe-bab8-4961-8a68-7a89c4969153 ('d32a8696-910b-4222-a364-1c00b5303fbb',) {} failed - stopping task >Thread-90::DEBUG::2012-12-11 11:55:55,723::task::1199::TaskManager.Task::(stop) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::stopping in state preparing (force False) >Thread-90::DEBUG::2012-12-11 11:55:55,723::task::978::TaskManager.Task::(_decref) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::ref 1 aborting True >Thread-90::INFO::2012-12-11 11:55:55,724::task::1157::TaskManager.Task::(prepare) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-90::DEBUG::2012-12-11 11:55:55,724::task::1162::TaskManager.Task::(prepare) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::Prepare: aborted: Unknown pool id, pool not connected >Thread-90::DEBUG::2012-12-11 11:55:55,724::task::978::TaskManager.Task::(_decref) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::ref 0 aborting True >Thread-90::DEBUG::2012-12-11 11:55:55,724::task::913::TaskManager.Task::(_doAbort) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::Task._doAbort: force False >Thread-90::DEBUG::2012-12-11 11:55:55,724::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2012-12-11 11:55:55,724::task::588::TaskManager.Task::(_updateState) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::moving from state preparing -> state aborting >Thread-90::DEBUG::2012-12-11 11:55:55,724::task::537::TaskManager.Task::(__state_aborting) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::_aborting: recover policy none >Thread-90::DEBUG::2012-12-11 11:55:55,724::task::588::TaskManager.Task::(_updateState) Task=`59477efe-bab8-4961-8a68-7a89c4969153`::moving from state aborting -> state failed >Thread-90::DEBUG::2012-12-11 11:55:55,725::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2012-12-11 11:55:55,725::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::ERROR::2012-12-11 11:55:55,725::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('d32a8696-910b-4222-a364-1c00b5303fbb',)", 'code': 309}} >Thread-91::DEBUG::2012-12-11 11:55:55,792::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-91::DEBUG::2012-12-11 11:55:55,792::task::588::TaskManager.Task::(_updateState) Task=`c4284245-0e95-43ec-b0eb-43e68b8c0693`::moving from state init -> state preparing >Thread-91::INFO::2012-12-11 11:55:55,792::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', hostID=1, scsiKey='d32a8696-910b-4222-a364-1c00b5303fbb', msdUUID='4c83c7d8-2e16-4607-a09c-abd89369b0c0', masterVersion=1, options=None) >Thread-91::DEBUG::2012-12-11 11:55:55,793::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`9e504f15-235c-4d57-b03f-5fc1158970ca`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-91::DEBUG::2012-12-11 11:55:55,793::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'exclusive' >Thread-91::DEBUG::2012-12-11 11:55:55,793::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'exclusive' (1 active user) >Thread-91::DEBUG::2012-12-11 11:55:55,793::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`9e504f15-235c-4d57-b03f-5fc1158970ca`::Granted request >Thread-91::INFO::2012-12-11 11:55:55,794::sp::625::Storage.StoragePool::(connect) Connect host #1 to the storage pool d32a8696-910b-4222-a364-1c00b5303fbb with master domain: 4c83c7d8-2e16-4607-a09c-abd89369b0c0 (ver = 1) >Thread-91::DEBUG::2012-12-11 11:55:55,794::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:55,794::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:55,794::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:55,794::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:55,795::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:55,795::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:55,795::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-91::DEBUG::2012-12-11 11:55:55,795::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-91::DEBUG::2012-12-11 11:55:55,795::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-91::DEBUG::2012-12-11 11:55:55,795::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-91::DEBUG::2012-12-11 11:55:55,796::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-91::DEBUG::2012-12-11 11:55:55,808::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-91::DEBUG::2012-12-11 11:55:55,808::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-91::DEBUG::2012-12-11 11:55:57,643::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-91::DEBUG::2012-12-11 11:55:57,673::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-91::DEBUG::2012-12-11 11:55:57,673::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,673::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,673::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,674::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,674::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,674::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,674::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-91::DEBUG::2012-12-11 11:55:57,674::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,675::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 4c83c7d8-2e16-4607-a09c-abd89369b0c0' (cwd None) >Thread-91::DEBUG::2012-12-11 11:55:57,692::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found\n'; <rc> = 5 >Thread-91::WARNING::2012-12-11 11:55:57,692::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found'] >Thread-91::DEBUG::2012-12-11 11:55:57,693::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,700::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-91::DEBUG::2012-12-11 11:55:57,700::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-91::DEBUG::2012-12-11 11:55:57,702::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=d32a8696-910b-4222-a364-1c00b5303fbb', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Master', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dfd43e5b9b700aadd22d3a8e46f80d8998686477'] >Thread-91::DEBUG::2012-12-11 11:55:57,703::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-91::WARNING::2012-12-11 11:55:57,703::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 4c83c7d8-2e16-4607-a09c-abd89369b0c0_imageNS already registered >Thread-91::WARNING::2012-12-11 11:55:57,703::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 4c83c7d8-2e16-4607-a09c-abd89369b0c0_volumeNS already registered >Thread-91::DEBUG::2012-12-11 11:55:57,703::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 verified, version 1 >Thread-91::DEBUG::2012-12-11 11:55:57,704::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-91::DEBUG::2012-12-11 11:55:57,704::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-91::DEBUG::2012-12-11 11:55:57,705::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=d32a8696-910b-4222-a364-1c00b5303fbb', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Master', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dfd43e5b9b700aadd22d3a8e46f80d8998686477'] >Thread-91::DEBUG::2012-12-11 11:55:57,706::sp::1495::Storage.StoragePool::(updateMonitoringThreads) sp `d32a8696-910b-4222-a364-1c00b5303fbb` started monitoring domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` >Thread-91::DEBUG::2012-12-11 11:55:57,707::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-91::DEBUG::2012-12-11 11:55:57,708::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-91::DEBUG::2012-12-11 11:55:57,709::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >Thread-91::DEBUG::2012-12-11 11:55:57,726::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-91::DEBUG::2012-12-11 11:55:57,727::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-91::INFO::2012-12-11 11:55:57,729::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0 to /rhev/data-center/d32a8696-910b-4222-a364-1c00b5303fbb/4c83c7d8-2e16-4607-a09c-abd89369b0c0 >Thread-91::INFO::2012-12-11 11:55:57,729::sp::1064::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0 to /rhev/data-center/d32a8696-910b-4222-a364-1c00b5303fbb/mastersd >Thread-91::INFO::2012-12-11 11:55:57,730::safelease::156::SANLock::(acquireHostId) Acquiring host id for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 (id: 1) >Thread-102::DEBUG::2012-12-11 11:56:03,516::task::588::TaskManager.Task::(_updateState) Task=`b2f9be14-c9b3-4035-9423-aa8a8555ecde`::moving from state init -> state preparing >Thread-102::INFO::2012-12-11 11:56:03,516::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-102::INFO::2012-12-11 11:56:03,516::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-102::DEBUG::2012-12-11 11:56:03,516::task::1172::TaskManager.Task::(prepare) Task=`b2f9be14-c9b3-4035-9423-aa8a8555ecde`::finished: {} >Thread-102::DEBUG::2012-12-11 11:56:03,516::task::588::TaskManager.Task::(_updateState) Task=`b2f9be14-c9b3-4035-9423-aa8a8555ecde`::moving from state preparing -> state finished >Thread-102::DEBUG::2012-12-11 11:56:03,517::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-102::DEBUG::2012-12-11 11:56:03,517::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2012-12-11 11:56:03,517::task::978::TaskManager.Task::(_decref) Task=`b2f9be14-c9b3-4035-9423-aa8a8555ecde`::ref 0 aborting False >Thread-108::DEBUG::2012-12-11 11:56:13,846::task::588::TaskManager.Task::(_updateState) Task=`16420189-15de-4bd8-87c8-05a4802860ae`::moving from state init -> state preparing >Thread-108::INFO::2012-12-11 11:56:13,846::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2012-12-11 11:56:13,846::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-108::DEBUG::2012-12-11 11:56:13,846::task::1172::TaskManager.Task::(prepare) Task=`16420189-15de-4bd8-87c8-05a4802860ae`::finished: {} >Thread-108::DEBUG::2012-12-11 11:56:13,846::task::588::TaskManager.Task::(_updateState) Task=`16420189-15de-4bd8-87c8-05a4802860ae`::moving from state preparing -> state finished >Thread-108::DEBUG::2012-12-11 11:56:13,846::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2012-12-11 11:56:13,846::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2012-12-11 11:56:13,847::task::978::TaskManager.Task::(_decref) Task=`16420189-15de-4bd8-87c8-05a4802860ae`::ref 0 aborting False >Thread-91::DEBUG::2012-12-11 11:56:18,732::safelease::166::SANLock::(acquireHostId) Host id for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 successfully acquired (id: 1) >Thread-91::DEBUG::2012-12-11 11:56:18,733::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-91::DEBUG::2012-12-11 11:56:18,733::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-91::DEBUG::2012-12-11 11:56:18,733::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-91::DEBUG::2012-12-11 11:56:18,733::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-91::INFO::2012-12-11 11:56:18,733::logUtils::39::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-91::DEBUG::2012-12-11 11:56:18,734::task::1172::TaskManager.Task::(prepare) Task=`c4284245-0e95-43ec-b0eb-43e68b8c0693`::finished: True >Thread-91::DEBUG::2012-12-11 11:56:18,734::task::588::TaskManager.Task::(_updateState) Task=`c4284245-0e95-43ec-b0eb-43e68b8c0693`::moving from state preparing -> state finished >Thread-91::DEBUG::2012-12-11 11:56:18,734::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-91::DEBUG::2012-12-11 11:56:18,734::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-91::DEBUG::2012-12-11 11:56:18,734::task::978::TaskManager.Task::(_decref) Task=`c4284245-0e95-43ec-b0eb-43e68b8c0693`::ref 0 aborting False >Thread-112::DEBUG::2012-12-11 11:56:18,774::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-112::DEBUG::2012-12-11 11:56:18,775::task::588::TaskManager.Task::(_updateState) Task=`1a47a76b-8ed7-4ec7-a32c-28ca3459c29e`::moving from state init -> state preparing >Thread-112::INFO::2012-12-11 11:56:18,775::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-112::INFO::2012-12-11 11:56:18,775::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-112::DEBUG::2012-12-11 11:56:18,775::task::1172::TaskManager.Task::(prepare) Task=`1a47a76b-8ed7-4ec7-a32c-28ca3459c29e`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-112::DEBUG::2012-12-11 11:56:18,775::task::588::TaskManager.Task::(_updateState) Task=`1a47a76b-8ed7-4ec7-a32c-28ca3459c29e`::moving from state preparing -> state finished >Thread-112::DEBUG::2012-12-11 11:56:18,775::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-112::DEBUG::2012-12-11 11:56:18,775::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-112::DEBUG::2012-12-11 11:56:18,776::task::978::TaskManager.Task::(_decref) Task=`1a47a76b-8ed7-4ec7-a32c-28ca3459c29e`::ref 0 aborting False >Thread-113::DEBUG::2012-12-11 11:56:18,862::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-113::DEBUG::2012-12-11 11:56:18,862::task::588::TaskManager.Task::(_updateState) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::moving from state init -> state preparing >Thread-113::INFO::2012-12-11 11:56:18,862::logUtils::37::dispatcher::(wrapper) Run and protect: spmStart(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', prevID=-1, prevLVER='-1', recoveryMode=None, scsiFencing='false', maxHostID=250, domVersion='3', options=None) >Thread-113::DEBUG::2012-12-11 11:56:18,863::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`847542d9-36b4-4e2a-b5d2-9cbd057c9a39`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-113::DEBUG::2012-12-11 11:56:18,863::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'exclusive' >Thread-113::DEBUG::2012-12-11 11:56:18,863::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'exclusive' (1 active user) >Thread-113::DEBUG::2012-12-11 11:56:18,863::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`847542d9-36b4-4e2a-b5d2-9cbd057c9a39`::Granted request >Thread-113::DEBUG::2012-12-11 11:56:18,864::task::817::TaskManager.Task::(resourceAcquired) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (exclusive) >Thread-113::DEBUG::2012-12-11 11:56:18,864::task::978::TaskManager.Task::(_decref) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::ref 1 aborting False >Thread-113::INFO::2012-12-11 11:56:18,864::logUtils::39::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >Thread-113::DEBUG::2012-12-11 11:56:18,864::task::1167::TaskManager.Task::(prepare) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::Prepare: 1 jobs exist, move to acquiring >Thread-113::DEBUG::2012-12-11 11:56:18,864::task::588::TaskManager.Task::(_updateState) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::moving from state preparing -> state acquiring >Thread-113::DEBUG::2012-12-11 11:56:18,864::task::588::TaskManager.Task::(_updateState) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::moving from state acquiring -> state queued >Thread-113::DEBUG::2012-12-11 11:56:18,864::taskManager::48::TaskManager::(_queueTask) queuing task: 8f52b0b2-2600-4c40-933e-248e745b05b0 >Thread-113::DEBUG::2012-12-11 11:56:18,865::taskManager::54::TaskManager::(_queueTask) task queued: 8f52b0b2-2600-4c40-933e-248e745b05b0 >Thread-2::DEBUG::2012-12-11 11:56:18,865::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-113::DEBUG::2012-12-11 11:56:18,865::task::1169::TaskManager.Task::(prepare) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::returning >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,865::threadPool::212::Misc.ThreadPool.WorkerThread::(run) Task: 8f52b0b2-2600-4c40-933e-248e745b05b0 running: <bound method Task.commit of <storage.task.Task instance at 0x7fd84c25f7e8>> with: None >Thread-113::DEBUG::2012-12-11 11:56:18,865::task::978::TaskManager.Task::(_decref) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::ref 0 aborting False >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,866::task::1181::TaskManager.Task::(commit) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::committing task: 8f52b0b2-2600-4c40-933e-248e745b05b0 >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,866::task::588::TaskManager.Task::(_updateState) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::moving from state queued -> state running >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,866::task::892::TaskManager.Task::(_runJobs) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7fd83800c050>> (args: (-1, '-1', 'false', 250, 3) kwargs: {}) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,866::task::317::TaskManager.Task::(run) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7fd83800c050>> (args: (-1, '-1', 'false', 250, 3) kwargs: {}) callback None >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,867::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,867::misc::1055::SamplingMethod::(__call__) Got in to sampling method >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,869::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=d32a8696-910b-4222-a364-1c00b5303fbb', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Master', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dfd43e5b9b700aadd22d3a8e46f80d8998686477'] >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,869::misc::1063::SamplingMethod::(__call__) Returning last result >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,870::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=d32a8696-910b-4222-a364-1c00b5303fbb', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Master', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dfd43e5b9b700aadd22d3a8e46f80d8998686477'] >8f52b0b2-2600-4c40-933e-248e745b05b0::INFO::2012-12-11 11:56:18,871::safelease::188::SANLock::(acquire) Acquiring cluster lock for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 (id: 1) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,902::safelease::213::SANLock::(acquire) Cluster lock for domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 successfully acquired (id: 1) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,903::sp::260::Storage.StoragePool::(startSpm) spm lock acquired successfully >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,904::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=d32a8696-910b-4222-a364-1c00b5303fbb', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Master', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=dfd43e5b9b700aadd22d3a8e46f80d8998686477'] >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,904::persistentDict::162::Storage.PersistentDict::(transaction) Starting transaction >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,904::persistentDict::168::Storage.PersistentDict::(transaction) Flushing changes >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,905::persistentDict::287::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=nfsStorageTest', 'POOL_DOMAINS=4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=d32a8696-910b-4222-a364-1c00b5303fbb', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'ROLE=Master', 'SDUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=ff434091664a352d6db7dc5090d8fb6380a90f4d'] >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,907::persistentDict::170::Storage.PersistentDict::(transaction) Finished transaction >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,907::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`ba0f9627-629d-46ab-b92d-6400c578bd42`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,907::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'exclusive' >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,907::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.upgrade_d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'exclusive' (1 active user) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,907::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.upgrade_d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`ba0f9627-629d-46ab-b92d-6400c578bd42`::Granted request >8f52b0b2-2600-4c40-933e-248e745b05b0::INFO::2012-12-11 11:56:18,908::sp::407::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,908::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0`ReqID=`7d799b2b-7665-414f-bc76-e05774769d33`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,908::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' for lock type 'exclusive' >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,908::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' is free. Now locking as 'exclusive' (1 active user) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,908::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0`ReqID=`7d799b2b-7665-414f-bc76-e05774769d33`::Granted request >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,909::sd::385::Storage.StorageDomain::(upgrade) Trying to upgrade domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` from version 3 to version 3 >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,909::sd::391::Storage.StorageDomain::(upgrade) No need to upgrade domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0`, leaving unchanged >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,909::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,909::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' (0 active users) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,909::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' is free, finding out if anyone is waiting for it. >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,909::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0', Clearing records. >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,910::sp::411::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,910::sp::418::Storage.StoragePool::(_upgradePool) Registering with state change event >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,910::sp::420::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,910::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_d32a8696-910b-4222-a364-1c00b5303fbb' >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,910::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,910::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.upgrade_d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,910::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >8f52b0b2-2600-4c40-933e-248e745b05b0::WARNING::2012-12-11 11:56:18,912::sd::564::Storage.StorageDomain::(createMasterTree) vms dir not found, creating (/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0/master/vms) >8f52b0b2-2600-4c40-933e-248e745b05b0::WARNING::2012-12-11 11:56:18,913::sd::569::Storage.StorageDomain::(createMasterTree) tasks dir not found, creating (/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873/4c83c7d8-2e16-4607-a09c-abd89369b0c0/master/tasks) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,924::sp::309::Storage.StoragePool::(startSpm) ended. >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,924::task::588::TaskManager.Task::(_updateState) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::moving from state running -> state finished >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,924::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,924::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,925::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,925::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,925::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,925::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,925::task::900::TaskManager.Task::(_runJobs) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::Task.run: exit - success: result >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,925::task::978::TaskManager.Task::(_decref) Task=`8f52b0b2-2600-4c40-933e-248e745b05b0`::ref 0 aborting False >8f52b0b2-2600-4c40-933e-248e745b05b0::DEBUG::2012-12-11 11:56:18,926::threadPool::67::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-114::DEBUG::2012-12-11 11:56:19,925::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-114::DEBUG::2012-12-11 11:56:19,925::task::588::TaskManager.Task::(_updateState) Task=`691632fd-7c86-4b08-b7e7-af96d1aa706c`::moving from state init -> state preparing >Thread-114::INFO::2012-12-11 11:56:19,925::logUtils::37::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='8f52b0b2-2600-4c40-933e-248e745b05b0', spUUID=None, options=None) >Thread-114::DEBUG::2012-12-11 11:56:19,926::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: 8f52b0b2-2600-4c40-933e-248e745b05b0 >Thread-114::DEBUG::2012-12-11 11:56:19,926::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '8f52b0b2-2600-4c40-933e-248e745b05b0'} >Thread-114::INFO::2012-12-11 11:56:19,926::logUtils::39::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '8f52b0b2-2600-4c40-933e-248e745b05b0'}} >Thread-114::DEBUG::2012-12-11 11:56:19,926::task::1172::TaskManager.Task::(prepare) Task=`691632fd-7c86-4b08-b7e7-af96d1aa706c`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '8f52b0b2-2600-4c40-933e-248e745b05b0'}} >Thread-114::DEBUG::2012-12-11 11:56:19,926::task::588::TaskManager.Task::(_updateState) Task=`691632fd-7c86-4b08-b7e7-af96d1aa706c`::moving from state preparing -> state finished >Thread-114::DEBUG::2012-12-11 11:56:19,926::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-114::DEBUG::2012-12-11 11:56:19,926::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-114::DEBUG::2012-12-11 11:56:19,927::task::978::TaskManager.Task::(_decref) Task=`691632fd-7c86-4b08-b7e7-af96d1aa706c`::ref 0 aborting False >Thread-115::DEBUG::2012-12-11 11:56:19,966::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-115::DEBUG::2012-12-11 11:56:19,967::task::588::TaskManager.Task::(_updateState) Task=`7910b23b-ace3-4ef8-b598-7391ee5ca2eb`::moving from state init -> state preparing >Thread-115::INFO::2012-12-11 11:56:19,967::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-115::INFO::2012-12-11 11:56:19,967::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-115::DEBUG::2012-12-11 11:56:19,967::task::1172::TaskManager.Task::(prepare) Task=`7910b23b-ace3-4ef8-b598-7391ee5ca2eb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-115::DEBUG::2012-12-11 11:56:19,967::task::588::TaskManager.Task::(_updateState) Task=`7910b23b-ace3-4ef8-b598-7391ee5ca2eb`::moving from state preparing -> state finished >Thread-115::DEBUG::2012-12-11 11:56:19,967::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2012-12-11 11:56:19,968::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2012-12-11 11:56:19,968::task::978::TaskManager.Task::(_decref) Task=`7910b23b-ace3-4ef8-b598-7391ee5ca2eb`::ref 0 aborting False >Thread-116::DEBUG::2012-12-11 11:56:20,037::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-116::DEBUG::2012-12-11 11:56:20,038::task::588::TaskManager.Task::(_updateState) Task=`68326bc8-beb8-40cf-b9a0-a98cfe4c9513`::moving from state init -> state preparing >Thread-116::INFO::2012-12-11 11:56:20,038::logUtils::37::dispatcher::(wrapper) Run and protect: clearTask(taskID='8f52b0b2-2600-4c40-933e-248e745b05b0', spUUID=None, options=None) >Thread-116::DEBUG::2012-12-11 11:56:20,038::taskManager::161::TaskManager::(clearTask) Entry. taskID: 8f52b0b2-2600-4c40-933e-248e745b05b0 >Thread-116::DEBUG::2012-12-11 11:56:20,038::taskManager::166::TaskManager::(clearTask) Return. >Thread-116::INFO::2012-12-11 11:56:20,038::logUtils::39::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-116::DEBUG::2012-12-11 11:56:20,038::task::1172::TaskManager.Task::(prepare) Task=`68326bc8-beb8-40cf-b9a0-a98cfe4c9513`::finished: None >Thread-116::DEBUG::2012-12-11 11:56:20,039::task::588::TaskManager.Task::(_updateState) Task=`68326bc8-beb8-40cf-b9a0-a98cfe4c9513`::moving from state preparing -> state finished >Thread-116::DEBUG::2012-12-11 11:56:20,039::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-116::DEBUG::2012-12-11 11:56:20,039::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-116::DEBUG::2012-12-11 11:56:20,039::task::978::TaskManager.Task::(_decref) Task=`68326bc8-beb8-40cf-b9a0-a98cfe4c9513`::ref 0 aborting False >Thread-118::DEBUG::2012-12-11 11:56:20,160::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-118::DEBUG::2012-12-11 11:56:20,160::task::588::TaskManager.Task::(_updateState) Task=`c1e6531b-3fb8-4473-a8a5-29d91b1414e9`::moving from state init -> state preparing >Thread-118::INFO::2012-12-11 11:56:20,161::logUtils::37::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='4c83c7d8-2e16-4607-a09c-abd89369b0c0', spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-118::DEBUG::2012-12-11 11:56:20,161::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`433d59d6-f749-4c00-b06c-2377def0f949`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-118::DEBUG::2012-12-11 11:56:20,161::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'exclusive' >Thread-118::DEBUG::2012-12-11 11:56:20,161::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'exclusive' (1 active user) >Thread-118::DEBUG::2012-12-11 11:56:20,161::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`433d59d6-f749-4c00-b06c-2377def0f949`::Granted request >Thread-118::DEBUG::2012-12-11 11:56:20,162::task::817::TaskManager.Task::(resourceAcquired) Task=`c1e6531b-3fb8-4473-a8a5-29d91b1414e9`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (exclusive) >Thread-118::DEBUG::2012-12-11 11:56:20,162::task::978::TaskManager.Task::(_decref) Task=`c1e6531b-3fb8-4473-a8a5-29d91b1414e9`::ref 1 aborting False >Thread-118::DEBUG::2012-12-11 11:56:20,162::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0`ReqID=`10f5ecfe-9b21-4486-a75c-dd85880f6b30`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-118::DEBUG::2012-12-11 11:56:20,162::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' for lock type 'exclusive' >Thread-118::DEBUG::2012-12-11 11:56:20,162::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' is free. Now locking as 'exclusive' (1 active user) >Thread-118::DEBUG::2012-12-11 11:56:20,163::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0`ReqID=`10f5ecfe-9b21-4486-a75c-dd85880f6b30`::Granted request >Thread-118::DEBUG::2012-12-11 11:56:20,163::task::817::TaskManager.Task::(resourceAcquired) Task=`c1e6531b-3fb8-4473-a8a5-29d91b1414e9`::_resourcesAcquired: Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0 (exclusive) >Thread-118::DEBUG::2012-12-11 11:56:20,163::task::978::TaskManager.Task::(_decref) Task=`c1e6531b-3fb8-4473-a8a5-29d91b1414e9`::ref 1 aborting False >Thread-118::INFO::2012-12-11 11:56:20,163::sp::986::Storage.StoragePool::(activateSD) sdUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0 spUUID=d32a8696-910b-4222-a364-1c00b5303fbb >Thread-118::INFO::2012-12-11 11:56:20,163::logUtils::39::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-118::DEBUG::2012-12-11 11:56:20,163::task::1172::TaskManager.Task::(prepare) Task=`c1e6531b-3fb8-4473-a8a5-29d91b1414e9`::finished: None >Thread-118::DEBUG::2012-12-11 11:56:20,164::task::588::TaskManager.Task::(_updateState) Task=`c1e6531b-3fb8-4473-a8a5-29d91b1414e9`::moving from state preparing -> state finished >Thread-118::DEBUG::2012-12-11 11:56:20,164::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>, 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0': < ResourceRef 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0', isValid: 'True' obj: 'None'>} >Thread-118::DEBUG::2012-12-11 11:56:20,164::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-118::DEBUG::2012-12-11 11:56:20,164::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-118::DEBUG::2012-12-11 11:56:20,164::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-118::DEBUG::2012-12-11 11:56:20,164::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-118::DEBUG::2012-12-11 11:56:20,165::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-118::DEBUG::2012-12-11 11:56:20,165::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' >Thread-118::DEBUG::2012-12-11 11:56:20,165::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' (0 active users) >Thread-118::DEBUG::2012-12-11 11:56:20,165::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0' is free, finding out if anyone is waiting for it. >Thread-118::DEBUG::2012-12-11 11:56:20,165::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.4c83c7d8-2e16-4607-a09c-abd89369b0c0', Clearing records. >Thread-118::DEBUG::2012-12-11 11:56:20,165::task::978::TaskManager.Task::(_decref) Task=`c1e6531b-3fb8-4473-a8a5-29d91b1414e9`::ref 0 aborting False >Thread-119::DEBUG::2012-12-11 11:56:20,211::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-119::DEBUG::2012-12-11 11:56:20,212::task::588::TaskManager.Task::(_updateState) Task=`3339b32a-8abd-42fe-a885-af59f1c63ba4`::moving from state init -> state preparing >Thread-119::INFO::2012-12-11 11:56:20,212::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', hostID=1, scsiKey='d32a8696-910b-4222-a364-1c00b5303fbb', msdUUID='4c83c7d8-2e16-4607-a09c-abd89369b0c0', masterVersion=1, options=None) >Thread-119::DEBUG::2012-12-11 11:56:20,212::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`6305cf29-1c81-4862-9c3b-4261b8049dab`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-119::DEBUG::2012-12-11 11:56:20,212::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'shared' >Thread-119::DEBUG::2012-12-11 11:56:20,213::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'shared' (1 active user) >Thread-119::DEBUG::2012-12-11 11:56:20,213::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`6305cf29-1c81-4862-9c3b-4261b8049dab`::Granted request >Thread-119::DEBUG::2012-12-11 11:56:20,213::sp::1465::Storage.StoragePool::(getMasterDomain) Master domain 4c83c7d8-2e16-4607-a09c-abd89369b0c0 verified, version 1 >Thread-119::DEBUG::2012-12-11 11:56:20,213::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-119::DEBUG::2012-12-11 11:56:20,213::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-119::DEBUG::2012-12-11 11:56:20,214::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-119::DEBUG::2012-12-11 11:56:20,214::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-119::INFO::2012-12-11 11:56:20,214::logUtils::39::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: None >Thread-119::DEBUG::2012-12-11 11:56:20,214::task::1172::TaskManager.Task::(prepare) Task=`3339b32a-8abd-42fe-a885-af59f1c63ba4`::finished: None >Thread-119::DEBUG::2012-12-11 11:56:20,214::task::588::TaskManager.Task::(_updateState) Task=`3339b32a-8abd-42fe-a885-af59f1c63ba4`::moving from state preparing -> state finished >Thread-119::DEBUG::2012-12-11 11:56:20,214::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-119::DEBUG::2012-12-11 11:56:20,214::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-119::DEBUG::2012-12-11 11:56:20,214::task::978::TaskManager.Task::(_decref) Task=`3339b32a-8abd-42fe-a885-af59f1c63ba4`::ref 0 aborting False >Thread-120::DEBUG::2012-12-11 11:56:21,355::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-120::DEBUG::2012-12-11 11:56:21,355::task::588::TaskManager.Task::(_updateState) Task=`a5dc1097-1a1d-470e-aacf-1c4913e31315`::moving from state init -> state preparing >Thread-120::INFO::2012-12-11 11:56:21,355::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '52b83eb6-c76d-46a7-ab4a-d871f74ab84e', 'port': ''}], options=None) >Thread-120::INFO::2012-12-11 11:56:21,355::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '52b83eb6-c76d-46a7-ab4a-d871f74ab84e'}]} >Thread-120::DEBUG::2012-12-11 11:56:21,356::task::1172::TaskManager.Task::(prepare) Task=`a5dc1097-1a1d-470e-aacf-1c4913e31315`::finished: {'statuslist': [{'status': 0, 'id': '52b83eb6-c76d-46a7-ab4a-d871f74ab84e'}]} >Thread-120::DEBUG::2012-12-11 11:56:21,356::task::588::TaskManager.Task::(_updateState) Task=`a5dc1097-1a1d-470e-aacf-1c4913e31315`::moving from state preparing -> state finished >Thread-120::DEBUG::2012-12-11 11:56:21,356::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-120::DEBUG::2012-12-11 11:56:21,356::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-120::DEBUG::2012-12-11 11:56:21,356::task::978::TaskManager.Task::(_decref) Task=`a5dc1097-1a1d-470e-aacf-1c4913e31315`::ref 0 aborting False >Thread-121::DEBUG::2012-12-11 11:56:21,425::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-121::DEBUG::2012-12-11 11:56:21,425::task::588::TaskManager.Task::(_updateState) Task=`a645d6fa-e325-489b-8ea4-d85a86b389a6`::moving from state init -> state preparing >Thread-121::INFO::2012-12-11 11:56:21,425::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '52b83eb6-c76d-46a7-ab4a-d871f74ab84e', 'port': ''}], options=None) >Thread-121::DEBUG::2012-12-11 11:56:21,429::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-121::DEBUG::2012-12-11 11:56:21,429::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-121::DEBUG::2012-12-11 11:56:21,429::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-121::DEBUG::2012-12-11 11:56:21,429::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-121::DEBUG::2012-12-11 11:56:21,430::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-121::DEBUG::2012-12-11 11:56:21,430::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-121::INFO::2012-12-11 11:56:21,430::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '52b83eb6-c76d-46a7-ab4a-d871f74ab84e'}]} >Thread-121::DEBUG::2012-12-11 11:56:21,430::task::1172::TaskManager.Task::(prepare) Task=`a645d6fa-e325-489b-8ea4-d85a86b389a6`::finished: {'statuslist': [{'status': 0, 'id': '52b83eb6-c76d-46a7-ab4a-d871f74ab84e'}]} >Thread-121::DEBUG::2012-12-11 11:56:21,430::task::588::TaskManager.Task::(_updateState) Task=`a645d6fa-e325-489b-8ea4-d85a86b389a6`::moving from state preparing -> state finished >Thread-121::DEBUG::2012-12-11 11:56:21,430::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-121::DEBUG::2012-12-11 11:56:21,431::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-121::DEBUG::2012-12-11 11:56:21,431::task::978::TaskManager.Task::(_decref) Task=`a645d6fa-e325-489b-8ea4-d85a86b389a6`::ref 0 aborting False >Thread-122::DEBUG::2012-12-11 11:56:21,477::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-122::DEBUG::2012-12-11 11:56:21,477::task::588::TaskManager.Task::(_updateState) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::moving from state init -> state preparing >Thread-122::INFO::2012-12-11 11:56:21,478::logUtils::37::dispatcher::(wrapper) Run and protect: attachStorageDomain(sdUUID='ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-122::DEBUG::2012-12-11 11:56:21,478::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`42919943-29d0-4a85-991f-e7ff87da87c9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-122::DEBUG::2012-12-11 11:56:21,478::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'exclusive' >Thread-122::DEBUG::2012-12-11 11:56:21,478::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'exclusive' (1 active user) >Thread-122::DEBUG::2012-12-11 11:56:21,479::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`42919943-29d0-4a85-991f-e7ff87da87c9`::Granted request >Thread-122::DEBUG::2012-12-11 11:56:21,479::task::817::TaskManager.Task::(resourceAcquired) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (exclusive) >Thread-122::DEBUG::2012-12-11 11:56:21,479::task::978::TaskManager.Task::(_decref) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::ref 1 aborting False >Thread-122::DEBUG::2012-12-11 11:56:21,480::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f`ReqID=`cf858134-1329-4168-9662-d7fd0bc88c0d`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-122::DEBUG::2012-12-11 11:56:21,480::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' for lock type 'exclusive' >Thread-122::DEBUG::2012-12-11 11:56:21,480::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' is free. Now locking as 'exclusive' (1 active user) >Thread-122::DEBUG::2012-12-11 11:56:21,480::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f`ReqID=`cf858134-1329-4168-9662-d7fd0bc88c0d`::Granted request >Thread-122::DEBUG::2012-12-11 11:56:21,480::task::817::TaskManager.Task::(resourceAcquired) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::_resourcesAcquired: Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f (exclusive) >Thread-122::DEBUG::2012-12-11 11:56:21,480::task::978::TaskManager.Task::(_decref) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::ref 1 aborting False >Thread-122::INFO::2012-12-11 11:56:21,481::sp::909::Storage.StoragePool::(attachSD) sdUUID=ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f spUUID=d32a8696-910b-4222-a364-1c00b5303fbb >Thread-122::DEBUG::2012-12-11 11:56:21,484::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-122::DEBUG::2012-12-11 11:56:21,484::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-122::DEBUG::2012-12-11 11:56:21,484::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-122::DEBUG::2012-12-11 11:56:21,485::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-122::DEBUG::2012-12-11 11:56:21,485::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-122::DEBUG::2012-12-11 11:56:21,497::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-122::DEBUG::2012-12-11 11:56:21,498::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-122::DEBUG::2012-12-11 11:56:23,332::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-122::DEBUG::2012-12-11 11:56:23,362::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-122::DEBUG::2012-12-11 11:56:23,362::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-122::DEBUG::2012-12-11 11:56:23,362::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-122::DEBUG::2012-12-11 11:56:23,363::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-122::DEBUG::2012-12-11 11:56:23,363::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-122::DEBUG::2012-12-11 11:56:23,363::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-122::DEBUG::2012-12-11 11:56:23,363::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-122::DEBUG::2012-12-11 11:56:23,363::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-122::DEBUG::2012-12-11 11:56:23,364::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-122::DEBUG::2012-12-11 11:56:23,364::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' (cwd None) >Thread-122::DEBUG::2012-12-11 11:56:23,382::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f" not found\n'; <rc> = 5 >Thread-122::WARNING::2012-12-11 11:56:23,382::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f" not found'] >Thread-122::DEBUG::2012-12-11 11:56:23,383::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-122::DEBUG::2012-12-11 11:56:23,389::fileSD::107::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__11__520428/ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f >Thread-122::DEBUG::2012-12-11 11:56:23,389::persistentDict::185::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-122::DEBUG::2012-12-11 11:56:23,392::persistentDict::226::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfsStorageTest_data_domain_1', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_11_520428', 'ROLE=Regular', 'SDUUID=ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=8a4c40f57bff06295914b5a2f2768cf01edec779'] >Thread-122::DEBUG::2012-12-11 11:56:23,393::fileSD::420::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-122::WARNING::2012-12-11 11:56:23,393::sd::312::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f_imageNS already registered >Thread-122::WARNING::2012-12-11 11:56:23,393::sd::318::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f_volumeNS already registered >Thread-122::INFO::2012-12-11 11:56:23,394::safelease::188::SANLock::(acquire) Acquiring cluster lock for domain ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f (id: 1) >Thread-122::ERROR::2012-12-11 11:56:23,394::task::853::TaskManager.Task::(_setError) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 861, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 38, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 960, in attachStorageDomain > pool.attachSD(sdUUID) > File "/usr/share/vdsm/storage/securable.py", line 63, in wrapper > return f(self, *args, **kwargs) > File "/usr/share/vdsm/storage/sp.py", line 919, in attachSD > dom.acquireClusterLock(self.id) > File "/usr/share/vdsm/storage/sd.py", line 429, in acquireClusterLock > self._clusterLock.acquire(hostID) > File "/usr/share/vdsm/storage/safelease.py", line 205, in acquire > "Cannot acquire cluster lock", str(e)) >AcquireLockFailure: Cannot obtain lock: "id=ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f, rc=28, out=Cannot acquire cluster lock, err=(28, 'Sanlock resource not acquired', 'No space left on device')" >Thread-122::DEBUG::2012-12-11 11:56:23,395::task::872::TaskManager.Task::(_run) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::Task._run: ee6adc28-58c0-458c-a4cc-17eaa838b4e9 ('ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', 'd32a8696-910b-4222-a364-1c00b5303fbb') {} failed - stopping task >Thread-122::DEBUG::2012-12-11 11:56:23,395::task::1199::TaskManager.Task::(stop) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::stopping in state preparing (force False) >Thread-122::DEBUG::2012-12-11 11:56:23,396::task::978::TaskManager.Task::(_decref) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::ref 1 aborting True >Thread-122::INFO::2012-12-11 11:56:23,396::task::1157::TaskManager.Task::(prepare) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::aborting: Task is aborted: 'Cannot obtain lock' - code 651 >Thread-122::DEBUG::2012-12-11 11:56:23,396::task::1162::TaskManager.Task::(prepare) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::Prepare: aborted: Cannot obtain lock >Thread-122::DEBUG::2012-12-11 11:56:23,396::task::978::TaskManager.Task::(_decref) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::ref 0 aborting True >Thread-122::DEBUG::2012-12-11 11:56:23,396::task::913::TaskManager.Task::(_doAbort) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::Task._doAbort: force False >Thread-122::DEBUG::2012-12-11 11:56:23,396::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2012-12-11 11:56:23,396::task::588::TaskManager.Task::(_updateState) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::moving from state preparing -> state aborting >Thread-122::DEBUG::2012-12-11 11:56:23,397::task::537::TaskManager.Task::(__state_aborting) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::_aborting: recover policy none >Thread-122::DEBUG::2012-12-11 11:56:23,397::task::588::TaskManager.Task::(_updateState) Task=`ee6adc28-58c0-458c-a4cc-17eaa838b4e9`::moving from state aborting -> state failed >Thread-122::DEBUG::2012-12-11 11:56:23,397::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>, 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f': < ResourceRef 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', isValid: 'True' obj: 'None'>} >Thread-122::DEBUG::2012-12-11 11:56:23,397::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2012-12-11 11:56:23,397::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-122::DEBUG::2012-12-11 11:56:23,397::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-122::DEBUG::2012-12-11 11:56:23,398::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-122::DEBUG::2012-12-11 11:56:23,398::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-122::DEBUG::2012-12-11 11:56:23,398::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' >Thread-122::DEBUG::2012-12-11 11:56:23,398::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' (0 active users) >Thread-122::DEBUG::2012-12-11 11:56:23,398::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f' is free, finding out if anyone is waiting for it. >Thread-122::DEBUG::2012-12-11 11:56:23,398::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f', Clearing records. >Thread-122::ERROR::2012-12-11 11:56:23,399::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': 'Cannot obtain lock: "id=ebb36e55-5d97-4ca1-abc4-cb98c6c82b4f, rc=28, out=Cannot acquire cluster lock, err=(28, \'Sanlock resource not acquired\', \'No space left on device\')"', 'code': 651}} >Thread-127::DEBUG::2012-12-11 11:56:24,266::task::588::TaskManager.Task::(_updateState) Task=`09eb0469-bad8-4fb1-b16b-ec7a8a51d77c`::moving from state init -> state preparing >Thread-127::INFO::2012-12-11 11:56:24,267::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-127::WARNING::2012-12-11 11:56:24,267::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:4c83c7d8-2e16-4607-a09c-abd89369b0c0 {'finish': 1355219777.730047, 'mdasize': 0, 'result': {'delay': '0.00148701667786', 'lastCheck': 1355219777.730047, 'code': 0, 'valid': True}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': True}, 'diskfree': '913253531648', 'mdafree': 0, 'disktotal': '1074600869888', 'mdathreshold': True} >Thread-127::INFO::2012-12-11 11:56:24,267::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0.00148701667786', 'lastCheck': 1355219777.730047, 'code': 355, 'valid': True}} >Thread-127::DEBUG::2012-12-11 11:56:24,268::task::1172::TaskManager.Task::(prepare) Task=`09eb0469-bad8-4fb1-b16b-ec7a8a51d77c`::finished: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0.00148701667786', 'lastCheck': 1355219777.730047, 'code': 355, 'valid': True}} >Thread-127::DEBUG::2012-12-11 11:56:24,268::task::588::TaskManager.Task::(_updateState) Task=`09eb0469-bad8-4fb1-b16b-ec7a8a51d77c`::moving from state preparing -> state finished >Thread-127::DEBUG::2012-12-11 11:56:24,268::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-127::DEBUG::2012-12-11 11:56:24,268::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-127::DEBUG::2012-12-11 11:56:24,268::task::978::TaskManager.Task::(_decref) Task=`09eb0469-bad8-4fb1-b16b-ec7a8a51d77c`::ref 0 aborting False >Thread-96::DEBUG::2012-12-11 11:56:27,734::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873 >Thread-96::DEBUG::2012-12-11 11:56:27,737::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-96::ERROR::2012-12-11 11:56:27,752::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873: device is busy\n') >Thread-96::DEBUG::2012-12-11 11:56:27,753::domainMonitor::130::Storage.DomainMonitor::(_monitorDomain) Domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` changed its status to Invalid >Thread-130::DEBUG::2012-12-11 11:56:27,753::misc::1175::Event.Storage.DomainMonitor.onDomainConnectivityStateChange::(_emit) Emitting event >Thread-130::DEBUG::2012-12-11 11:56:27,754::misc::1186::Event.Storage.DomainMonitor.onDomainConnectivityStateChange::(_emit) Calling registered method `<functools.partial object at 0x7fd84c31a998>` >Thread-130::DEBUG::2012-12-11 11:56:27,754::misc::1196::Event.Storage.DomainMonitor.onDomainConnectivityStateChange::(_emit) Event emitted >Thread-133::DEBUG::2012-12-11 11:56:30,199::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-133::DEBUG::2012-12-11 11:56:30,200::task::588::TaskManager.Task::(_updateState) Task=`e9b4e581-b9a0-482a-a5e0-309174cd7b18`::moving from state init -> state preparing >Thread-133::INFO::2012-12-11 11:56:30,200::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-133::INFO::2012-12-11 11:56:30,200::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-133::DEBUG::2012-12-11 11:56:30,200::task::1172::TaskManager.Task::(prepare) Task=`e9b4e581-b9a0-482a-a5e0-309174cd7b18`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-133::DEBUG::2012-12-11 11:56:30,200::task::588::TaskManager.Task::(_updateState) Task=`e9b4e581-b9a0-482a-a5e0-309174cd7b18`::moving from state preparing -> state finished >Thread-133::DEBUG::2012-12-11 11:56:30,200::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2012-12-11 11:56:30,201::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2012-12-11 11:56:30,201::task::978::TaskManager.Task::(_decref) Task=`e9b4e581-b9a0-482a-a5e0-309174cd7b18`::ref 0 aborting False >Thread-134::DEBUG::2012-12-11 11:56:30,277::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-134::DEBUG::2012-12-11 11:56:30,278::task::588::TaskManager.Task::(_updateState) Task=`13c18aad-6897-4076-8639-2c2366d2c766`::moving from state init -> state preparing >Thread-134::INFO::2012-12-11 11:56:30,278::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-134::DEBUG::2012-12-11 11:56:30,278::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`875a3ff0-fb13-4404-bcc0-600519adc544`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-134::DEBUG::2012-12-11 11:56:30,278::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'shared' >Thread-134::DEBUG::2012-12-11 11:56:30,279::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'shared' (1 active user) >Thread-134::DEBUG::2012-12-11 11:56:30,279::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`875a3ff0-fb13-4404-bcc0-600519adc544`::Granted request >Thread-134::DEBUG::2012-12-11 11:56:30,279::task::817::TaskManager.Task::(resourceAcquired) Task=`13c18aad-6897-4076-8639-2c2366d2c766`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (shared) >Thread-134::DEBUG::2012-12-11 11:56:30,279::task::978::TaskManager.Task::(_decref) Task=`13c18aad-6897-4076-8639-2c2366d2c766`::ref 1 aborting False >Thread-134::INFO::2012-12-11 11:56:30,283::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-134::DEBUG::2012-12-11 11:56:30,283::task::1172::TaskManager.Task::(prepare) Task=`13c18aad-6897-4076-8639-2c2366d2c766`::finished: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-134::DEBUG::2012-12-11 11:56:30,283::task::588::TaskManager.Task::(_updateState) Task=`13c18aad-6897-4076-8639-2c2366d2c766`::moving from state preparing -> state finished >Thread-134::DEBUG::2012-12-11 11:56:30,284::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >Thread-134::DEBUG::2012-12-11 11:56:30,284::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2012-12-11 11:56:30,284::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-134::DEBUG::2012-12-11 11:56:30,284::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-134::DEBUG::2012-12-11 11:56:30,284::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-134::DEBUG::2012-12-11 11:56:30,284::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-134::DEBUG::2012-12-11 11:56:30,285::task::978::TaskManager.Task::(_decref) Task=`13c18aad-6897-4076-8639-2c2366d2c766`::ref 0 aborting False >Thread-137::DEBUG::2012-12-11 11:56:34,778::task::588::TaskManager.Task::(_updateState) Task=`27593bd1-f828-4852-b903-a6a621f8db28`::moving from state init -> state preparing >Thread-137::INFO::2012-12-11 11:56:34,779::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-137::WARNING::2012-12-11 11:56:34,779::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:4c83c7d8-2e16-4607-a09c-abd89369b0c0 {'finish': 1355219787.753182, 'mdasize': 0, 'result': {'delay': '0', 'lastCheck': 1355219787.753182, 'code': 200, 'valid': False}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': False}, 'diskfree': None, 'mdafree': 0, 'disktotal': None, 'mdathreshold': True} >Thread-137::INFO::2012-12-11 11:56:34,780::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219787.753182, 'code': 200, 'valid': False}} >Thread-137::DEBUG::2012-12-11 11:56:34,780::task::1172::TaskManager.Task::(prepare) Task=`27593bd1-f828-4852-b903-a6a621f8db28`::finished: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219787.753182, 'code': 200, 'valid': False}} >Thread-137::DEBUG::2012-12-11 11:56:34,780::task::588::TaskManager.Task::(_updateState) Task=`27593bd1-f828-4852-b903-a6a621f8db28`::moving from state preparing -> state finished >Thread-137::DEBUG::2012-12-11 11:56:34,780::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-137::DEBUG::2012-12-11 11:56:34,780::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-137::DEBUG::2012-12-11 11:56:34,780::task::978::TaskManager.Task::(_decref) Task=`27593bd1-f828-4852-b903-a6a621f8db28`::ref 0 aborting False >Thread-96::DEBUG::2012-12-11 11:56:37,759::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873 >Thread-96::DEBUG::2012-12-11 11:56:37,762::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-96::ERROR::2012-12-11 11:56:37,781::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873: device is busy\n') >Thread-141::DEBUG::2012-12-11 11:56:40,325::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-141::DEBUG::2012-12-11 11:56:40,325::task::588::TaskManager.Task::(_updateState) Task=`18695475-42ab-4a10-8145-6e17c1eacf60`::moving from state init -> state preparing >Thread-141::INFO::2012-12-11 11:56:40,326::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-141::INFO::2012-12-11 11:56:40,326::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-141::DEBUG::2012-12-11 11:56:40,326::task::1172::TaskManager.Task::(prepare) Task=`18695475-42ab-4a10-8145-6e17c1eacf60`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-141::DEBUG::2012-12-11 11:56:40,326::task::588::TaskManager.Task::(_updateState) Task=`18695475-42ab-4a10-8145-6e17c1eacf60`::moving from state preparing -> state finished >Thread-141::DEBUG::2012-12-11 11:56:40,326::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-141::DEBUG::2012-12-11 11:56:40,326::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-141::DEBUG::2012-12-11 11:56:40,327::task::978::TaskManager.Task::(_decref) Task=`18695475-42ab-4a10-8145-6e17c1eacf60`::ref 0 aborting False >Thread-142::DEBUG::2012-12-11 11:56:40,395::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-142::DEBUG::2012-12-11 11:56:40,395::task::588::TaskManager.Task::(_updateState) Task=`d6ad942a-6392-41cb-965d-bc47752b8cf0`::moving from state init -> state preparing >Thread-142::INFO::2012-12-11 11:56:40,396::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-142::DEBUG::2012-12-11 11:56:40,396::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`d087626c-aad0-4f19-b0d5-3447e73e7942`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-142::DEBUG::2012-12-11 11:56:40,396::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'shared' >Thread-142::DEBUG::2012-12-11 11:56:40,396::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'shared' (1 active user) >Thread-142::DEBUG::2012-12-11 11:56:40,396::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`d087626c-aad0-4f19-b0d5-3447e73e7942`::Granted request >Thread-142::DEBUG::2012-12-11 11:56:40,397::task::817::TaskManager.Task::(resourceAcquired) Task=`d6ad942a-6392-41cb-965d-bc47752b8cf0`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (shared) >Thread-142::DEBUG::2012-12-11 11:56:40,397::task::978::TaskManager.Task::(_decref) Task=`d6ad942a-6392-41cb-965d-bc47752b8cf0`::ref 1 aborting False >Thread-142::INFO::2012-12-11 11:56:40,401::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-142::DEBUG::2012-12-11 11:56:40,401::task::1172::TaskManager.Task::(prepare) Task=`d6ad942a-6392-41cb-965d-bc47752b8cf0`::finished: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-142::DEBUG::2012-12-11 11:56:40,401::task::588::TaskManager.Task::(_updateState) Task=`d6ad942a-6392-41cb-965d-bc47752b8cf0`::moving from state preparing -> state finished >Thread-142::DEBUG::2012-12-11 11:56:40,401::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >Thread-142::DEBUG::2012-12-11 11:56:40,401::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-142::DEBUG::2012-12-11 11:56:40,402::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-142::DEBUG::2012-12-11 11:56:40,402::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-142::DEBUG::2012-12-11 11:56:40,402::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-142::DEBUG::2012-12-11 11:56:40,402::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-142::DEBUG::2012-12-11 11:56:40,402::task::978::TaskManager.Task::(_decref) Task=`d6ad942a-6392-41cb-965d-bc47752b8cf0`::ref 0 aborting False >Thread-145::DEBUG::2012-12-11 11:56:45,202::task::588::TaskManager.Task::(_updateState) Task=`1da713fd-aa5b-4e68-8e79-afd97d2ea7ed`::moving from state init -> state preparing >Thread-145::INFO::2012-12-11 11:56:45,202::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-145::WARNING::2012-12-11 11:56:45,202::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:4c83c7d8-2e16-4607-a09c-abd89369b0c0 {'finish': 1355219797.781691, 'mdasize': 0, 'result': {'delay': '0', 'lastCheck': 1355219797.781691, 'code': 200, 'valid': False}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': False}, 'diskfree': None, 'mdafree': 0, 'disktotal': None, 'mdathreshold': True} >Thread-145::INFO::2012-12-11 11:56:45,203::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219797.781691, 'code': 200, 'valid': False}} >Thread-145::DEBUG::2012-12-11 11:56:45,203::task::1172::TaskManager.Task::(prepare) Task=`1da713fd-aa5b-4e68-8e79-afd97d2ea7ed`::finished: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219797.781691, 'code': 200, 'valid': False}} >Thread-145::DEBUG::2012-12-11 11:56:45,203::task::588::TaskManager.Task::(_updateState) Task=`1da713fd-aa5b-4e68-8e79-afd97d2ea7ed`::moving from state preparing -> state finished >Thread-145::DEBUG::2012-12-11 11:56:45,203::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-145::DEBUG::2012-12-11 11:56:45,203::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-145::DEBUG::2012-12-11 11:56:45,203::task::978::TaskManager.Task::(_decref) Task=`1da713fd-aa5b-4e68-8e79-afd97d2ea7ed`::ref 0 aborting False >Thread-96::DEBUG::2012-12-11 11:56:47,786::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873 >Thread-96::DEBUG::2012-12-11 11:56:47,789::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-96::ERROR::2012-12-11 11:56:47,803::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873: device is busy\n') >Thread-149::DEBUG::2012-12-11 11:56:50,445::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-149::DEBUG::2012-12-11 11:56:50,445::task::588::TaskManager.Task::(_updateState) Task=`95ff376e-f464-47ae-abc8-3b4d489cfba3`::moving from state init -> state preparing >Thread-149::INFO::2012-12-11 11:56:50,446::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-149::INFO::2012-12-11 11:56:50,446::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-149::DEBUG::2012-12-11 11:56:50,446::task::1172::TaskManager.Task::(prepare) Task=`95ff376e-f464-47ae-abc8-3b4d489cfba3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-149::DEBUG::2012-12-11 11:56:50,446::task::588::TaskManager.Task::(_updateState) Task=`95ff376e-f464-47ae-abc8-3b4d489cfba3`::moving from state preparing -> state finished >Thread-149::DEBUG::2012-12-11 11:56:50,446::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-149::DEBUG::2012-12-11 11:56:50,447::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-149::DEBUG::2012-12-11 11:56:50,447::task::978::TaskManager.Task::(_decref) Task=`95ff376e-f464-47ae-abc8-3b4d489cfba3`::ref 0 aborting False >Thread-150::DEBUG::2012-12-11 11:56:50,487::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-150::DEBUG::2012-12-11 11:56:50,487::task::588::TaskManager.Task::(_updateState) Task=`c39c99e1-4495-42cb-9d14-b9e6b0a7033d`::moving from state init -> state preparing >Thread-150::INFO::2012-12-11 11:56:50,487::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-150::DEBUG::2012-12-11 11:56:50,487::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`de688ea8-44b8-4b47-a36d-c5a732c92a79`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-150::DEBUG::2012-12-11 11:56:50,487::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'shared' >Thread-150::DEBUG::2012-12-11 11:56:50,488::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'shared' (1 active user) >Thread-150::DEBUG::2012-12-11 11:56:50,488::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`de688ea8-44b8-4b47-a36d-c5a732c92a79`::Granted request >Thread-150::DEBUG::2012-12-11 11:56:50,488::task::817::TaskManager.Task::(resourceAcquired) Task=`c39c99e1-4495-42cb-9d14-b9e6b0a7033d`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (shared) >Thread-150::DEBUG::2012-12-11 11:56:50,488::task::978::TaskManager.Task::(_decref) Task=`c39c99e1-4495-42cb-9d14-b9e6b0a7033d`::ref 1 aborting False >Thread-150::INFO::2012-12-11 11:56:50,492::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-150::DEBUG::2012-12-11 11:56:50,492::task::1172::TaskManager.Task::(prepare) Task=`c39c99e1-4495-42cb-9d14-b9e6b0a7033d`::finished: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-150::DEBUG::2012-12-11 11:56:50,493::task::588::TaskManager.Task::(_updateState) Task=`c39c99e1-4495-42cb-9d14-b9e6b0a7033d`::moving from state preparing -> state finished >Thread-150::DEBUG::2012-12-11 11:56:50,493::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >Thread-150::DEBUG::2012-12-11 11:56:50,493::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-150::DEBUG::2012-12-11 11:56:50,493::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-150::DEBUG::2012-12-11 11:56:50,493::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-150::DEBUG::2012-12-11 11:56:50,493::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-150::DEBUG::2012-12-11 11:56:50,493::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-150::DEBUG::2012-12-11 11:56:50,494::task::978::TaskManager.Task::(_decref) Task=`c39c99e1-4495-42cb-9d14-b9e6b0a7033d`::ref 0 aborting False >Thread-153::DEBUG::2012-12-11 11:56:55,574::task::588::TaskManager.Task::(_updateState) Task=`6fa0fd93-a3a2-48fe-b282-96879d9ec9da`::moving from state init -> state preparing >Thread-153::INFO::2012-12-11 11:56:55,574::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-153::WARNING::2012-12-11 11:56:55,574::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:4c83c7d8-2e16-4607-a09c-abd89369b0c0 {'finish': 1355219807.803906, 'mdasize': 0, 'result': {'delay': '0', 'lastCheck': 1355219807.803906, 'code': 200, 'valid': False}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': False}, 'diskfree': None, 'mdafree': 0, 'disktotal': None, 'mdathreshold': True} >Thread-153::INFO::2012-12-11 11:56:55,575::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219807.803906, 'code': 200, 'valid': False}} >Thread-153::DEBUG::2012-12-11 11:56:55,575::task::1172::TaskManager.Task::(prepare) Task=`6fa0fd93-a3a2-48fe-b282-96879d9ec9da`::finished: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219807.803906, 'code': 200, 'valid': False}} >Thread-153::DEBUG::2012-12-11 11:56:55,575::task::588::TaskManager.Task::(_updateState) Task=`6fa0fd93-a3a2-48fe-b282-96879d9ec9da`::moving from state preparing -> state finished >Thread-153::DEBUG::2012-12-11 11:56:55,575::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-153::DEBUG::2012-12-11 11:56:55,575::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-153::DEBUG::2012-12-11 11:56:55,575::task::978::TaskManager.Task::(_decref) Task=`6fa0fd93-a3a2-48fe-b282-96879d9ec9da`::ref 0 aborting False >Thread-96::DEBUG::2012-12-11 11:56:57,808::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873 >Thread-96::DEBUG::2012-12-11 11:56:57,811::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-96::ERROR::2012-12-11 11:56:57,825::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873: device is busy\n') >Thread-157::DEBUG::2012-12-11 11:57:00,548::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-157::DEBUG::2012-12-11 11:57:00,548::task::588::TaskManager.Task::(_updateState) Task=`1e864955-6715-42bd-b010-baa7ff9af44c`::moving from state init -> state preparing >Thread-157::INFO::2012-12-11 11:57:00,549::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-157::INFO::2012-12-11 11:57:00,549::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-157::DEBUG::2012-12-11 11:57:00,549::task::1172::TaskManager.Task::(prepare) Task=`1e864955-6715-42bd-b010-baa7ff9af44c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-157::DEBUG::2012-12-11 11:57:00,549::task::588::TaskManager.Task::(_updateState) Task=`1e864955-6715-42bd-b010-baa7ff9af44c`::moving from state preparing -> state finished >Thread-157::DEBUG::2012-12-11 11:57:00,549::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-157::DEBUG::2012-12-11 11:57:00,549::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-157::DEBUG::2012-12-11 11:57:00,550::task::978::TaskManager.Task::(_decref) Task=`1e864955-6715-42bd-b010-baa7ff9af44c`::ref 0 aborting False >Thread-158::DEBUG::2012-12-11 11:57:00,627::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-158::DEBUG::2012-12-11 11:57:00,627::task::588::TaskManager.Task::(_updateState) Task=`ab65f58a-f84d-4140-90b8-1908888e345b`::moving from state init -> state preparing >Thread-158::INFO::2012-12-11 11:57:00,628::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-158::DEBUG::2012-12-11 11:57:00,628::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`e410111a-ab8e-4784-af96-c3aa3a72723a`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-158::DEBUG::2012-12-11 11:57:00,628::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'shared' >Thread-158::DEBUG::2012-12-11 11:57:00,628::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'shared' (1 active user) >Thread-158::DEBUG::2012-12-11 11:57:00,628::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`e410111a-ab8e-4784-af96-c3aa3a72723a`::Granted request >Thread-158::DEBUG::2012-12-11 11:57:00,629::task::817::TaskManager.Task::(resourceAcquired) Task=`ab65f58a-f84d-4140-90b8-1908888e345b`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (shared) >Thread-158::DEBUG::2012-12-11 11:57:00,629::task::978::TaskManager.Task::(_decref) Task=`ab65f58a-f84d-4140-90b8-1908888e345b`::ref 1 aborting False >Thread-158::INFO::2012-12-11 11:57:00,633::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-158::DEBUG::2012-12-11 11:57:00,633::task::1172::TaskManager.Task::(prepare) Task=`ab65f58a-f84d-4140-90b8-1908888e345b`::finished: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-158::DEBUG::2012-12-11 11:57:00,633::task::588::TaskManager.Task::(_updateState) Task=`ab65f58a-f84d-4140-90b8-1908888e345b`::moving from state preparing -> state finished >Thread-158::DEBUG::2012-12-11 11:57:00,633::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >Thread-158::DEBUG::2012-12-11 11:57:00,633::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-158::DEBUG::2012-12-11 11:57:00,634::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-158::DEBUG::2012-12-11 11:57:00,634::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-158::DEBUG::2012-12-11 11:57:00,634::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-158::DEBUG::2012-12-11 11:57:00,634::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-158::DEBUG::2012-12-11 11:57:00,634::task::978::TaskManager.Task::(_decref) Task=`ab65f58a-f84d-4140-90b8-1908888e345b`::ref 0 aborting False >Thread-161::DEBUG::2012-12-11 11:57:06,023::task::588::TaskManager.Task::(_updateState) Task=`3cd6ed8b-fadc-4c64-8659-dea3255c7e4d`::moving from state init -> state preparing >Thread-161::INFO::2012-12-11 11:57:06,024::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-161::WARNING::2012-12-11 11:57:06,024::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:4c83c7d8-2e16-4607-a09c-abd89369b0c0 {'finish': 1355219817.826449, 'mdasize': 0, 'result': {'delay': '0', 'lastCheck': 1355219817.826449, 'code': 200, 'valid': False}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': False}, 'diskfree': None, 'mdafree': 0, 'disktotal': None, 'mdathreshold': True} >Thread-161::INFO::2012-12-11 11:57:06,024::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219817.826449, 'code': 200, 'valid': False}} >Thread-161::DEBUG::2012-12-11 11:57:06,024::task::1172::TaskManager.Task::(prepare) Task=`3cd6ed8b-fadc-4c64-8659-dea3255c7e4d`::finished: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219817.826449, 'code': 200, 'valid': False}} >Thread-161::DEBUG::2012-12-11 11:57:06,024::task::588::TaskManager.Task::(_updateState) Task=`3cd6ed8b-fadc-4c64-8659-dea3255c7e4d`::moving from state preparing -> state finished >Thread-161::DEBUG::2012-12-11 11:57:06,024::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-161::DEBUG::2012-12-11 11:57:06,025::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-161::DEBUG::2012-12-11 11:57:06,025::task::978::TaskManager.Task::(_decref) Task=`3cd6ed8b-fadc-4c64-8659-dea3255c7e4d`::ref 0 aborting False >Thread-96::DEBUG::2012-12-11 11:57:07,830::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873 >Thread-96::DEBUG::2012-12-11 11:57:07,834::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-96::ERROR::2012-12-11 11:57:07,847::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873: device is busy\n') >Thread-165::DEBUG::2012-12-11 11:57:10,692::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-165::DEBUG::2012-12-11 11:57:10,692::task::588::TaskManager.Task::(_updateState) Task=`056e8e08-bb43-47d0-b2a6-0a1324b8cf05`::moving from state init -> state preparing >Thread-165::INFO::2012-12-11 11:57:10,692::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-165::INFO::2012-12-11 11:57:10,693::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-165::DEBUG::2012-12-11 11:57:10,693::task::1172::TaskManager.Task::(prepare) Task=`056e8e08-bb43-47d0-b2a6-0a1324b8cf05`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-165::DEBUG::2012-12-11 11:57:10,693::task::588::TaskManager.Task::(_updateState) Task=`056e8e08-bb43-47d0-b2a6-0a1324b8cf05`::moving from state preparing -> state finished >Thread-165::DEBUG::2012-12-11 11:57:10,693::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-165::DEBUG::2012-12-11 11:57:10,693::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-165::DEBUG::2012-12-11 11:57:10,693::task::978::TaskManager.Task::(_decref) Task=`056e8e08-bb43-47d0-b2a6-0a1324b8cf05`::ref 0 aborting False >Thread-166::DEBUG::2012-12-11 11:57:10,719::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-166::DEBUG::2012-12-11 11:57:10,719::task::588::TaskManager.Task::(_updateState) Task=`5e12acb2-f978-48c3-ad9e-269a26d50229`::moving from state init -> state preparing >Thread-166::INFO::2012-12-11 11:57:10,719::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-166::DEBUG::2012-12-11 11:57:10,719::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`496ddffb-59ef-4abc-91f9-03db7b79a6f6`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-166::DEBUG::2012-12-11 11:57:10,720::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'shared' >Thread-166::DEBUG::2012-12-11 11:57:10,720::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'shared' (1 active user) >Thread-166::DEBUG::2012-12-11 11:57:10,720::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`496ddffb-59ef-4abc-91f9-03db7b79a6f6`::Granted request >Thread-166::DEBUG::2012-12-11 11:57:10,720::task::817::TaskManager.Task::(resourceAcquired) Task=`5e12acb2-f978-48c3-ad9e-269a26d50229`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (shared) >Thread-166::DEBUG::2012-12-11 11:57:10,720::task::978::TaskManager.Task::(_decref) Task=`5e12acb2-f978-48c3-ad9e-269a26d50229`::ref 1 aborting False >Thread-166::INFO::2012-12-11 11:57:10,724::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-166::DEBUG::2012-12-11 11:57:10,724::task::1172::TaskManager.Task::(prepare) Task=`5e12acb2-f978-48c3-ad9e-269a26d50229`::finished: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-166::DEBUG::2012-12-11 11:57:10,725::task::588::TaskManager.Task::(_updateState) Task=`5e12acb2-f978-48c3-ad9e-269a26d50229`::moving from state preparing -> state finished >Thread-166::DEBUG::2012-12-11 11:57:10,725::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >Thread-166::DEBUG::2012-12-11 11:57:10,725::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-166::DEBUG::2012-12-11 11:57:10,725::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-166::DEBUG::2012-12-11 11:57:10,725::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-166::DEBUG::2012-12-11 11:57:10,725::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-166::DEBUG::2012-12-11 11:57:10,726::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-166::DEBUG::2012-12-11 11:57:10,726::task::978::TaskManager.Task::(_decref) Task=`5e12acb2-f978-48c3-ad9e-269a26d50229`::ref 0 aborting False >Thread-169::DEBUG::2012-12-11 11:57:16,381::task::588::TaskManager.Task::(_updateState) Task=`08efce10-0899-4249-b2ef-108e0690e9ee`::moving from state init -> state preparing >Thread-169::INFO::2012-12-11 11:57:16,381::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-169::WARNING::2012-12-11 11:57:16,381::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:4c83c7d8-2e16-4607-a09c-abd89369b0c0 {'finish': 1355219827.848209, 'mdasize': 0, 'result': {'delay': '0', 'lastCheck': 1355219827.848209, 'code': 200, 'valid': False}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': False}, 'diskfree': None, 'mdafree': 0, 'disktotal': None, 'mdathreshold': True} >Thread-169::INFO::2012-12-11 11:57:16,381::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219827.848209, 'code': 200, 'valid': False}} >Thread-169::DEBUG::2012-12-11 11:57:16,381::task::1172::TaskManager.Task::(prepare) Task=`08efce10-0899-4249-b2ef-108e0690e9ee`::finished: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219827.848209, 'code': 200, 'valid': False}} >Thread-169::DEBUG::2012-12-11 11:57:16,381::task::588::TaskManager.Task::(_updateState) Task=`08efce10-0899-4249-b2ef-108e0690e9ee`::moving from state preparing -> state finished >Thread-169::DEBUG::2012-12-11 11:57:16,382::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-169::DEBUG::2012-12-11 11:57:16,382::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-169::DEBUG::2012-12-11 11:57:16,382::task::978::TaskManager.Task::(_decref) Task=`08efce10-0899-4249-b2ef-108e0690e9ee`::ref 0 aborting False >Thread-96::DEBUG::2012-12-11 11:57:17,852::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873 >Thread-96::DEBUG::2012-12-11 11:57:17,855::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-96::ERROR::2012-12-11 11:57:17,869::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873: device is busy\n') >Thread-173::DEBUG::2012-12-11 11:57:20,798::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-173::DEBUG::2012-12-11 11:57:20,799::task::588::TaskManager.Task::(_updateState) Task=`7eb97cdf-a9a0-414b-a5d0-494aa0e679fb`::moving from state init -> state preparing >Thread-173::INFO::2012-12-11 11:57:20,799::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-173::INFO::2012-12-11 11:57:20,799::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-173::DEBUG::2012-12-11 11:57:20,799::task::1172::TaskManager.Task::(prepare) Task=`7eb97cdf-a9a0-414b-a5d0-494aa0e679fb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-173::DEBUG::2012-12-11 11:57:20,799::task::588::TaskManager.Task::(_updateState) Task=`7eb97cdf-a9a0-414b-a5d0-494aa0e679fb`::moving from state preparing -> state finished >Thread-173::DEBUG::2012-12-11 11:57:20,800::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-173::DEBUG::2012-12-11 11:57:20,800::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-173::DEBUG::2012-12-11 11:57:20,800::task::978::TaskManager.Task::(_decref) Task=`7eb97cdf-a9a0-414b-a5d0-494aa0e679fb`::ref 0 aborting False >Thread-174::DEBUG::2012-12-11 11:57:20,840::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-174::DEBUG::2012-12-11 11:57:20,841::task::588::TaskManager.Task::(_updateState) Task=`279d3ebb-dfa0-44ea-8f6d-1d053dd9d867`::moving from state init -> state preparing >Thread-174::INFO::2012-12-11 11:57:20,841::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-174::DEBUG::2012-12-11 11:57:20,841::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`f3ed63ed-5db7-439f-99a8-5f1f8eef69d8`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-174::DEBUG::2012-12-11 11:57:20,841::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'shared' >Thread-174::DEBUG::2012-12-11 11:57:20,842::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'shared' (1 active user) >Thread-174::DEBUG::2012-12-11 11:57:20,842::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`f3ed63ed-5db7-439f-99a8-5f1f8eef69d8`::Granted request >Thread-174::DEBUG::2012-12-11 11:57:20,842::task::817::TaskManager.Task::(resourceAcquired) Task=`279d3ebb-dfa0-44ea-8f6d-1d053dd9d867`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (shared) >Thread-174::DEBUG::2012-12-11 11:57:20,842::task::978::TaskManager.Task::(_decref) Task=`279d3ebb-dfa0-44ea-8f6d-1d053dd9d867`::ref 1 aborting False >Thread-174::INFO::2012-12-11 11:57:20,846::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-174::DEBUG::2012-12-11 11:57:20,846::task::1172::TaskManager.Task::(prepare) Task=`279d3ebb-dfa0-44ea-8f6d-1d053dd9d867`::finished: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-174::DEBUG::2012-12-11 11:57:20,846::task::588::TaskManager.Task::(_updateState) Task=`279d3ebb-dfa0-44ea-8f6d-1d053dd9d867`::moving from state preparing -> state finished >Thread-174::DEBUG::2012-12-11 11:57:20,847::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >Thread-174::DEBUG::2012-12-11 11:57:20,847::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-174::DEBUG::2012-12-11 11:57:20,847::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-174::DEBUG::2012-12-11 11:57:20,847::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-174::DEBUG::2012-12-11 11:57:20,847::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-174::DEBUG::2012-12-11 11:57:20,847::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-174::DEBUG::2012-12-11 11:57:20,848::task::978::TaskManager.Task::(_decref) Task=`279d3ebb-dfa0-44ea-8f6d-1d053dd9d867`::ref 0 aborting False >Thread-177::DEBUG::2012-12-11 11:57:26,715::task::588::TaskManager.Task::(_updateState) Task=`7f3ba404-9903-4ee1-b80f-f5d4602acceb`::moving from state init -> state preparing >Thread-177::INFO::2012-12-11 11:57:26,715::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-177::WARNING::2012-12-11 11:57:26,715::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:4c83c7d8-2e16-4607-a09c-abd89369b0c0 {'finish': 1355219837.870132, 'mdasize': 0, 'result': {'delay': '0', 'lastCheck': 1355219837.870132, 'code': 200, 'valid': False}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': False}, 'diskfree': None, 'mdafree': 0, 'disktotal': None, 'mdathreshold': True} >Thread-177::INFO::2012-12-11 11:57:26,715::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219837.870132, 'code': 200, 'valid': False}} >Thread-177::DEBUG::2012-12-11 11:57:26,715::task::1172::TaskManager.Task::(prepare) Task=`7f3ba404-9903-4ee1-b80f-f5d4602acceb`::finished: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219837.870132, 'code': 200, 'valid': False}} >Thread-177::DEBUG::2012-12-11 11:57:26,715::task::588::TaskManager.Task::(_updateState) Task=`7f3ba404-9903-4ee1-b80f-f5d4602acceb`::moving from state preparing -> state finished >Thread-177::DEBUG::2012-12-11 11:57:26,716::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-177::DEBUG::2012-12-11 11:57:26,716::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-177::DEBUG::2012-12-11 11:57:26,716::task::978::TaskManager.Task::(_decref) Task=`7f3ba404-9903-4ee1-b80f-f5d4602acceb`::ref 0 aborting False >Thread-96::DEBUG::2012-12-11 11:57:27,874::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873 >Thread-96::DEBUG::2012-12-11 11:57:27,877::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-96::ERROR::2012-12-11 11:57:27,891::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873: device is busy\n') >Thread-180::DEBUG::2012-12-11 11:57:30,884::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-180::DEBUG::2012-12-11 11:57:30,885::task::588::TaskManager.Task::(_updateState) Task=`cb21c2d7-0b5f-4ac7-a3dd-0da4ab5572d5`::moving from state init -> state preparing >Thread-180::INFO::2012-12-11 11:57:30,885::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-180::INFO::2012-12-11 11:57:30,885::logUtils::39::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-180::DEBUG::2012-12-11 11:57:30,885::task::1172::TaskManager.Task::(prepare) Task=`cb21c2d7-0b5f-4ac7-a3dd-0da4ab5572d5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 0}} >Thread-180::DEBUG::2012-12-11 11:57:30,886::task::588::TaskManager.Task::(_updateState) Task=`cb21c2d7-0b5f-4ac7-a3dd-0da4ab5572d5`::moving from state preparing -> state finished >Thread-180::DEBUG::2012-12-11 11:57:30,886::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-180::DEBUG::2012-12-11 11:57:30,886::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-180::DEBUG::2012-12-11 11:57:30,886::task::978::TaskManager.Task::(_decref) Task=`cb21c2d7-0b5f-4ac7-a3dd-0da4ab5572d5`::ref 0 aborting False >Thread-182::DEBUG::2012-12-11 11:57:30,940::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-182::DEBUG::2012-12-11 11:57:30,941::task::588::TaskManager.Task::(_updateState) Task=`e82b7bdc-0990-48f7-8fde-ae179eb82778`::moving from state init -> state preparing >Thread-182::INFO::2012-12-11 11:57:30,941::logUtils::37::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-182::DEBUG::2012-12-11 11:57:30,941::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`327bbe50-8442-48df-bddd-e08d15273271`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-182::DEBUG::2012-12-11 11:57:30,941::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'shared' >Thread-182::DEBUG::2012-12-11 11:57:30,942::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'shared' (1 active user) >Thread-182::DEBUG::2012-12-11 11:57:30,942::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`327bbe50-8442-48df-bddd-e08d15273271`::Granted request >Thread-182::DEBUG::2012-12-11 11:57:30,942::task::817::TaskManager.Task::(resourceAcquired) Task=`e82b7bdc-0990-48f7-8fde-ae179eb82778`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (shared) >Thread-182::DEBUG::2012-12-11 11:57:30,942::task::978::TaskManager.Task::(_decref) Task=`e82b7bdc-0990-48f7-8fde-ae179eb82778`::ref 1 aborting False >Thread-182::INFO::2012-12-11 11:57:30,946::logUtils::39::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-182::DEBUG::2012-12-11 11:57:30,946::task::1172::TaskManager.Task::(prepare) Task=`e82b7bdc-0990-48f7-8fde-ae179eb82778`::finished: {'info': {'spm_id': 1, 'master_uuid': '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 'name': 'nfsStorageTest', 'version': '3', 'domains': '4c83c7d8-2e16-4607-a09c-abd89369b0c0:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 0}, 'dominfo': {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'status': 'Active', 'alerts': []}}} >Thread-182::DEBUG::2012-12-11 11:57:30,946::task::588::TaskManager.Task::(_updateState) Task=`e82b7bdc-0990-48f7-8fde-ae179eb82778`::moving from state preparing -> state finished >Thread-182::DEBUG::2012-12-11 11:57:30,946::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >Thread-182::DEBUG::2012-12-11 11:57:30,947::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-182::DEBUG::2012-12-11 11:57:30,947::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-182::DEBUG::2012-12-11 11:57:30,947::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-182::DEBUG::2012-12-11 11:57:30,947::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-182::DEBUG::2012-12-11 11:57:30,947::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-182::DEBUG::2012-12-11 11:57:30,947::task::978::TaskManager.Task::(_decref) Task=`e82b7bdc-0990-48f7-8fde-ae179eb82778`::ref 0 aborting False >Thread-185::DEBUG::2012-12-11 11:57:37,076::task::588::TaskManager.Task::(_updateState) Task=`9fca3615-39bb-4ac1-9c5c-bffb994d96be`::moving from state init -> state preparing >Thread-185::INFO::2012-12-11 11:57:37,077::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-185::WARNING::2012-12-11 11:57:37,077::hsm::3061::Storage.HSM::(repoStats) repoStats detected invalid master:4c83c7d8-2e16-4607-a09c-abd89369b0c0 {'finish': 1355219847.891967, 'mdasize': 0, 'result': {'delay': '0', 'lastCheck': 1355219847.891967, 'code': 200, 'valid': False}, 'mdavalid': True, 'masterValidate': {'mount': False, 'valid': False}, 'diskfree': None, 'mdafree': 0, 'disktotal': None, 'mdathreshold': True} >Thread-185::INFO::2012-12-11 11:57:37,077::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219847.891967, 'code': 200, 'valid': False}} >Thread-185::DEBUG::2012-12-11 11:57:37,077::task::1172::TaskManager.Task::(prepare) Task=`9fca3615-39bb-4ac1-9c5c-bffb994d96be`::finished: {'4c83c7d8-2e16-4607-a09c-abd89369b0c0': {'delay': '0', 'lastCheck': 1355219847.891967, 'code': 200, 'valid': False}} >Thread-185::DEBUG::2012-12-11 11:57:37,077::task::588::TaskManager.Task::(_updateState) Task=`9fca3615-39bb-4ac1-9c5c-bffb994d96be`::moving from state preparing -> state finished >Thread-185::DEBUG::2012-12-11 11:57:37,077::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-185::DEBUG::2012-12-11 11:57:37,077::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-185::DEBUG::2012-12-11 11:57:37,078::task::978::TaskManager.Task::(_decref) Task=`9fca3615-39bb-4ac1-9c5c-bffb994d96be`::ref 0 aborting False >Thread-96::DEBUG::2012-12-11 11:57:37,896::fileSD::406::Storage.StorageDomain::(selftest) Unmounting stale file system /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873 >Thread-96::DEBUG::2012-12-11 11:57:37,900::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/umount /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873' (cwd None) >Thread-96::ERROR::2012-12-11 11:57:37,914::domainMonitor::120::Storage.DomainMonitor::(_monitorDomain) Error while collecting domain `4c83c7d8-2e16-4607-a09c-abd89369b0c0` monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 103, in _monitorDomain > domain.selftest() > File "/usr/share/vdsm/storage/nfsSD.py", line 103, in selftest > fileSD.FileStorageDomain.selftest(self) > File "/usr/share/vdsm/storage/fileSD.py", line 407, in selftest > mount.getMountFromTarget(self.mountpoint).umount() > File "/usr/share/vdsm/storage/mount.py", line 218, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (16, ';umount.nfs: /rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873: device is busy\n') >MainThread::INFO::2012-12-11 11:57:39,091::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-11 11:57:39,092::task::588::TaskManager.Task::(_updateState) Task=`bd3bc86f-bc00-48fc-840b-8f1501ae4f68`::moving from state init -> state preparing >MainThread::DEBUG::2012-12-11 11:57:39,092::clientIF::180::vds::(prepareForShutdown) cannot run prepareForShutdown concurrently >MainThread::INFO::2012-12-11 11:57:39,092::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-11 11:57:39,093::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-11 11:57:39,132::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-11 11:57:39,132::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-11 11:57:39,133::task::1172::TaskManager.Task::(prepare) Task=`bd3bc86f-bc00-48fc-840b-8f1501ae4f68`::finished: None >MainThread::DEBUG::2012-12-11 11:57:39,133::task::588::TaskManager.Task::(_updateState) Task=`bd3bc86f-bc00-48fc-840b-8f1501ae4f68`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-11 11:57:39,133::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-11 11:57:39,133::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-11 11:57:39,133::task::978::TaskManager.Task::(_decref) Task=`bd3bc86f-bc00-48fc-840b-8f1501ae4f68`::ref 0 aborting False >MainThread::INFO::2012-12-11 11:57:39,134::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 14 other threads... >MainThread::INFO::2012-12-11 11:57:39,134::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 140567956342528)> >MainThread::INFO::2012-12-11 11:57:39,134::vdsm::78::vds::(run) <WorkerThread(8f52b0b2-2600-4c40-933e-248e745b05b0, started daemon 140567918581504)> >MainThread::INFO::2012-12-11 11:57:39,134::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 140567166056192)> >MainThread::INFO::2012-12-11 11:57:39,134::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 140567652570880)> >MainThread::INFO::2012-12-11 11:57:39,134::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 140567702927104)> >MainThread::INFO::2012-12-11 11:57:39,134::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-11 11:57:39,134::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 140567115699968)> >MainThread::INFO::2012-12-11 11:57:39,135::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 140567660963584)> >MainThread::INFO::2012-12-11 11:57:39,135::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 140567132485376)> >MainThread::INFO::2012-12-11 11:57:39,135::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 140567677748992)> >MainThread::INFO::2012-12-11 11:57:39,135::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 140567669356288)> >MainThread::INFO::2012-12-11 11:57:39,135::vdsm::78::vds::(run) <_MainThread(MainThread, started 140568186783552)> >MainThread::INFO::2012-12-11 11:57:39,135::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 140567686141696)> >MainThread::INFO::2012-12-11 11:57:39,135::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 140567926974208)> >MainThread::INFO::2012-12-11 11:57:39,135::vdsm::78::vds::(run) <Thread(Thread-26, started daemon 140566620792576)> >MainThread::INFO::2012-12-11 11:57:39,136::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 140567694534400)> >MainThread::INFO::2012-12-11 11:57:39,220::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-11 11:57:39,387::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-11 11:57:39,388::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-11 11:57:39,408::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 1933' (cwd None) >MainThread::DEBUG::2012-12-11 11:57:39,429::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 11:57:39,429::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-11 11:57:39,429::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 4b386cf2-b80a-4810-a76d-d0ecf6257971 2714' (cwd None) >MainThread::DEBUG::2012-12-11 11:57:39,564::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-11 11:57:39,565::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-11 11:57:39,565::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-11 11:57:39,565::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-11 11:57:39,565::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-11 11:57:39,565::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-11 11:57:39,566::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-11 11:57:41,434::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-11 11:57:41,439::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-11 11:57:41,440::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-11 11:57:41,440::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-11 11:57:41,460::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 11:57:41,461::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 11:57:41,461::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-11 11:57:41,477::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 11:57:41,477::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 11:57:41,477::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 11:57:41,478::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-11 11:57:41,492::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-11 11:57:41,493::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 11:57:41,493::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-11 11:57:41,508::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-11 11:57:41,508::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-11 11:57:41,509::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-11 11:57:41,509::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 11:57:41,510::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-11 11:57:41,510::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 11:57:41,511::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-11 11:57:41,518::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-11 11:57:41,521::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 11:57:41,521::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-11 11:57:41,525::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-11 11:57:41,526::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-11 11:57:41,528::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-11 11:57:41,529::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-14::DEBUG::2012-12-11 11:57:43,256::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-11 11:57:43,352::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-12::DEBUG::2012-12-11 11:57:43,359::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-14::DEBUG::2012-12-11 11:57:43,368::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:57:43,368::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-11 11:57:43,383::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:57:43,384::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-11 11:57:43,399::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:57:43,399::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-11 11:57:43,414::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:57:43,415::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-12::DEBUG::2012-12-11 11:57:43,421::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 11:57:43,422::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 11:57:43,422::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 11:57:43,422::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 11:57:43,422::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 11:57:43,422::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 11:57:43,423::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 11:57:43,423::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-11 11:57:43,423::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-11 11:57:43,427::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-11 11:57:43,427::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__10__163873', '/rhev/data-center/mnt/10.35.148.12:_vol_vfilers_nfs01_vm-18-155__nfs__2012__12__11__11__52__11__520428', '/rhev/data-center/mnt/10.35.16.27:_volumes_shafan_vm-18-155__nfs__20121211115212316558'] >Thread-12::DEBUG::2012-12-11 11:57:43,427::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-11 11:57:43,429::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-11 11:57:43,431::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 11:57:44,219::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '4.fc18.x86_64', 'buildtime': 1354623171.0, 'version': '3.6.9'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '1.fc18', 'buildtime': '1355100085', 'version': '0.10.2.2'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >Thread-16::DEBUG::2012-12-11 11:57:44,366::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-16::DEBUG::2012-12-11 11:57:44,366::task::588::TaskManager.Task::(_updateState) Task=`ff0d564c-2006-463a-8584-35dddc6ee988`::moving from state init -> state preparing >Thread-16::INFO::2012-12-11 11:57:44,369::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=1, spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5', 'port': ''}], options=None) >Thread-16::INFO::2012-12-11 11:57:44,369::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-16::DEBUG::2012-12-11 11:57:44,369::task::1172::TaskManager.Task::(prepare) Task=`ff0d564c-2006-463a-8584-35dddc6ee988`::finished: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-16::DEBUG::2012-12-11 11:57:44,370::task::588::TaskManager.Task::(_updateState) Task=`ff0d564c-2006-463a-8584-35dddc6ee988`::moving from state preparing -> state finished >Thread-16::DEBUG::2012-12-11 11:57:44,370::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-16::DEBUG::2012-12-11 11:57:44,370::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-16::DEBUG::2012-12-11 11:57:44,370::task::978::TaskManager.Task::(_decref) Task=`ff0d564c-2006-463a-8584-35dddc6ee988`::ref 0 aborting False >Thread-17::DEBUG::2012-12-11 11:57:44,441::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-17::DEBUG::2012-12-11 11:57:44,442::task::588::TaskManager.Task::(_updateState) Task=`1e7ec902-1cf9-48fa-b7a9-74af89d8809a`::moving from state init -> state preparing >Thread-17::INFO::2012-12-11 11:57:44,442::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', conList=[{'connection': '10.35.148.12:/vol/vfilers/nfs01/vm-18-155_nfs_2012_12_11_11_52_10_163873', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5', 'port': ''}], options=None) >Thread-17::DEBUG::2012-12-11 11:57:44,445::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2012-12-11 11:57:44,446::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2012-12-11 11:57:44,446::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2012-12-11 11:57:44,446::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::DEBUG::2012-12-11 11:57:44,446::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-17::DEBUG::2012-12-11 11:57:44,446::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-17::INFO::2012-12-11 11:57:44,447::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-17::DEBUG::2012-12-11 11:57:44,447::task::1172::TaskManager.Task::(prepare) Task=`1e7ec902-1cf9-48fa-b7a9-74af89d8809a`::finished: {'statuslist': [{'status': 0, 'id': '4203bf6c-8e6c-4ea9-a4c5-01c0090036c5'}]} >Thread-17::DEBUG::2012-12-11 11:57:44,447::task::588::TaskManager.Task::(_updateState) Task=`1e7ec902-1cf9-48fa-b7a9-74af89d8809a`::moving from state preparing -> state finished >Thread-17::DEBUG::2012-12-11 11:57:44,447::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2012-12-11 11:57:44,447::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2012-12-11 11:57:44,447::task::978::TaskManager.Task::(_decref) Task=`1e7ec902-1cf9-48fa-b7a9-74af89d8809a`::ref 0 aborting False >Thread-18::DEBUG::2012-12-11 11:57:44,487::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-18::DEBUG::2012-12-11 11:57:44,488::task::588::TaskManager.Task::(_updateState) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::moving from state init -> state preparing >Thread-18::INFO::2012-12-11 11:57:44,488::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', hostID=1, scsiKey='d32a8696-910b-4222-a364-1c00b5303fbb', msdUUID='4c83c7d8-2e16-4607-a09c-abd89369b0c0', masterVersion=1, options=None) >Thread-18::DEBUG::2012-12-11 11:57:44,488::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`e149496e-a4fa-4baa-808c-b519cf12dde4`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-18::DEBUG::2012-12-11 11:57:44,488::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'exclusive' >Thread-18::DEBUG::2012-12-11 11:57:44,489::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'exclusive' (1 active user) >Thread-18::DEBUG::2012-12-11 11:57:44,489::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`e149496e-a4fa-4baa-808c-b519cf12dde4`::Granted request >Thread-18::INFO::2012-12-11 11:57:44,489::sp::625::Storage.StoragePool::(connect) Connect host #1 to the storage pool d32a8696-910b-4222-a364-1c00b5303fbb with master domain: 4c83c7d8-2e16-4607-a09c-abd89369b0c0 (ver = 1) >Thread-18::DEBUG::2012-12-11 11:57:44,490::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:44,490::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:44,490::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:44,490::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:44,490::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:44,491::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:44,491::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-18::DEBUG::2012-12-11 11:57:44,491::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-18::DEBUG::2012-12-11 11:57:44,491::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-18::DEBUG::2012-12-11 11:57:44,491::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-18::DEBUG::2012-12-11 11:57:44,491::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-18::DEBUG::2012-12-11 11:57:44,504::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2012-12-11 11:57:44,504::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-18::DEBUG::2012-12-11 11:57:46,338::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-18::DEBUG::2012-12-11 11:57:46,378::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2012-12-11 11:57:46,378::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:46,379::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:46,379::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:46,379::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:46,379::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:46,379::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:46,380::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-18::DEBUG::2012-12-11 11:57:46,380::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:46,380::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 4c83c7d8-2e16-4607-a09c-abd89369b0c0' (cwd None) >Thread-18::DEBUG::2012-12-11 11:57:46,400::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = ' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found\n'; <rc> = 5 >Thread-18::WARNING::2012-12-11 11:57:46,401::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "4c83c7d8-2e16-4607-a09c-abd89369b0c0" not found'] >Thread-18::DEBUG::2012-12-11 11:57:46,401::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-18::DEBUG::2012-12-11 11:57:46,593::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-18::DEBUG::2012-12-11 11:57:46,593::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-18::DEBUG::2012-12-11 11:57:46,593::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-18::DEBUG::2012-12-11 11:57:46,594::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-18::ERROR::2012-12-11 11:57:46,594::task::853::TaskManager.Task::(_setError) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 861, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 38, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 817, in connectStoragePool > return self._connectStoragePool(spUUID, hostID, scsiKey, msdUUID, masterVersion, options) > File "/usr/share/vdsm/storage/hsm.py", line 859, in _connectStoragePool > res = pool.connect(hostID, scsiKey, msdUUID, masterVersion) > File "/usr/share/vdsm/storage/sp.py", line 641, in connect > self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion) > File "/usr/share/vdsm/storage/sp.py", line 1109, in __rebuild > self.masterDomain = self.getMasterDomain(msdUUID=msdUUID, masterVersion=masterVersion) > File "/usr/share/vdsm/storage/sp.py", line 1448, in getMasterDomain > raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID) >StoragePoolMasterNotFound: Cannot find master domain: 'spUUID=d32a8696-910b-4222-a364-1c00b5303fbb, msdUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0' >Thread-18::DEBUG::2012-12-11 11:57:46,596::task::872::TaskManager.Task::(_run) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::Task._run: 114d84ef-3d1d-45d2-ae8c-d2035742a6b9 ('d32a8696-910b-4222-a364-1c00b5303fbb', 1, 'd32a8696-910b-4222-a364-1c00b5303fbb', '4c83c7d8-2e16-4607-a09c-abd89369b0c0', 1) {} failed - stopping task >Thread-18::DEBUG::2012-12-11 11:57:46,596::task::1199::TaskManager.Task::(stop) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::stopping in state preparing (force False) >Thread-18::DEBUG::2012-12-11 11:57:46,596::task::978::TaskManager.Task::(_decref) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::ref 1 aborting True >Thread-18::INFO::2012-12-11 11:57:46,597::task::1157::TaskManager.Task::(prepare) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::aborting: Task is aborted: 'Cannot find master domain' - code 304 >Thread-18::DEBUG::2012-12-11 11:57:46,597::task::1162::TaskManager.Task::(prepare) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::Prepare: aborted: Cannot find master domain >Thread-18::DEBUG::2012-12-11 11:57:46,597::task::978::TaskManager.Task::(_decref) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::ref 0 aborting True >Thread-18::DEBUG::2012-12-11 11:57:46,597::task::913::TaskManager.Task::(_doAbort) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::Task._doAbort: force False >Thread-18::DEBUG::2012-12-11 11:57:46,597::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::DEBUG::2012-12-11 11:57:46,597::task::588::TaskManager.Task::(_updateState) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::moving from state preparing -> state aborting >Thread-18::DEBUG::2012-12-11 11:57:46,597::task::537::TaskManager.Task::(__state_aborting) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::_aborting: recover policy none >Thread-18::DEBUG::2012-12-11 11:57:46,598::task::588::TaskManager.Task::(_updateState) Task=`114d84ef-3d1d-45d2-ae8c-d2035742a6b9`::moving from state aborting -> state failed >Thread-18::DEBUG::2012-12-11 11:57:46,598::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-18::DEBUG::2012-12-11 11:57:46,598::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-18::ERROR::2012-12-11 11:57:46,598::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Cannot find master domain: 'spUUID=d32a8696-910b-4222-a364-1c00b5303fbb, msdUUID=4c83c7d8-2e16-4607-a09c-abd89369b0c0'", 'code': 304}} >Thread-23::DEBUG::2012-12-11 11:57:46,706::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-23::DEBUG::2012-12-11 11:57:46,707::task::588::TaskManager.Task::(_updateState) Task=`a54e8ca4-7dcb-47e1-a62a-170ffbd0a98b`::moving from state init -> state preparing >Thread-23::INFO::2012-12-11 11:57:46,707::logUtils::37::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-23::DEBUG::2012-12-11 11:57:46,707::taskManager::103::TaskManager::(getAllTasksStatuses) Entry. >Thread-23::DEBUG::2012-12-11 11:57:46,707::taskManager::112::TaskManager::(getAllTasksStatuses) Return: {} >Thread-23::INFO::2012-12-11 11:57:46,707::logUtils::39::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-23::DEBUG::2012-12-11 11:57:46,708::task::1172::TaskManager.Task::(prepare) Task=`a54e8ca4-7dcb-47e1-a62a-170ffbd0a98b`::finished: {'allTasksStatus': {}} >Thread-23::DEBUG::2012-12-11 11:57:46,708::task::588::TaskManager.Task::(_updateState) Task=`a54e8ca4-7dcb-47e1-a62a-170ffbd0a98b`::moving from state preparing -> state finished >Thread-23::DEBUG::2012-12-11 11:57:46,708::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-23::DEBUG::2012-12-11 11:57:46,708::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-23::DEBUG::2012-12-11 11:57:46,708::task::978::TaskManager.Task::(_decref) Task=`a54e8ca4-7dcb-47e1-a62a-170ffbd0a98b`::ref 0 aborting False >Thread-24::DEBUG::2012-12-11 11:57:46,767::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-24::DEBUG::2012-12-11 11:57:46,767::task::588::TaskManager.Task::(_updateState) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::moving from state init -> state preparing >Thread-24::INFO::2012-12-11 11:57:46,767::logUtils::37::dispatcher::(wrapper) Run and protect: spmStop(spUUID='d32a8696-910b-4222-a364-1c00b5303fbb', options=None) >Thread-24::DEBUG::2012-12-11 11:57:46,767::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`3399afaf-e812-4f32-aa6d-d8976103aea9`::Request was made in '/usr/share/vdsm/storage/resourceManager.py' line '485' at 'registerResource' >Thread-24::DEBUG::2012-12-11 11:57:46,768::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' for lock type 'exclusive' >Thread-24::DEBUG::2012-12-11 11:57:46,768::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free. Now locking as 'exclusive' (1 active user) >Thread-24::DEBUG::2012-12-11 11:57:46,768::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.d32a8696-910b-4222-a364-1c00b5303fbb`ReqID=`3399afaf-e812-4f32-aa6d-d8976103aea9`::Granted request >Thread-24::DEBUG::2012-12-11 11:57:46,768::task::817::TaskManager.Task::(resourceAcquired) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::_resourcesAcquired: Storage.d32a8696-910b-4222-a364-1c00b5303fbb (exclusive) >Thread-24::DEBUG::2012-12-11 11:57:46,768::task::978::TaskManager.Task::(_decref) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::ref 1 aborting False >Thread-24::ERROR::2012-12-11 11:57:46,768::task::853::TaskManager.Task::(_setError) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 861, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 38, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 523, in spmStop > pool = self.getPool(spUUID) > File "/usr/share/vdsm/storage/hsm.py", line 265, in getPool > raise se.StoragePoolUnknown(spUUID) >StoragePoolUnknown: Unknown pool id, pool not connected: ('d32a8696-910b-4222-a364-1c00b5303fbb',) >Thread-24::DEBUG::2012-12-11 11:57:46,769::task::872::TaskManager.Task::(_run) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::Task._run: 9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a ('d32a8696-910b-4222-a364-1c00b5303fbb',) {} failed - stopping task >Thread-24::DEBUG::2012-12-11 11:57:46,769::task::1199::TaskManager.Task::(stop) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::stopping in state preparing (force False) >Thread-24::DEBUG::2012-12-11 11:57:46,769::task::978::TaskManager.Task::(_decref) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::ref 1 aborting True >Thread-24::INFO::2012-12-11 11:57:46,769::task::1157::TaskManager.Task::(prepare) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-24::DEBUG::2012-12-11 11:57:46,769::task::1162::TaskManager.Task::(prepare) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::Prepare: aborted: Unknown pool id, pool not connected >Thread-24::DEBUG::2012-12-11 11:57:46,769::task::978::TaskManager.Task::(_decref) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::ref 0 aborting True >Thread-24::DEBUG::2012-12-11 11:57:46,770::task::913::TaskManager.Task::(_doAbort) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::Task._doAbort: force False >Thread-24::DEBUG::2012-12-11 11:57:46,770::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-24::DEBUG::2012-12-11 11:57:46,770::task::588::TaskManager.Task::(_updateState) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::moving from state preparing -> state aborting >Thread-24::DEBUG::2012-12-11 11:57:46,770::task::537::TaskManager.Task::(__state_aborting) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::_aborting: recover policy none >Thread-24::DEBUG::2012-12-11 11:57:46,770::task::588::TaskManager.Task::(_updateState) Task=`9dce6bd7-61fb-4f88-a34c-2bc1cdfcb36a`::moving from state aborting -> state failed >Thread-24::DEBUG::2012-12-11 11:57:46,770::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.d32a8696-910b-4222-a364-1c00b5303fbb': < ResourceRef 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', isValid: 'True' obj: 'None'>} >Thread-24::DEBUG::2012-12-11 11:57:46,770::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-24::DEBUG::2012-12-11 11:57:46,771::resourceManager::538::ResourceManager::(releaseResource) Trying to release resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' >Thread-24::DEBUG::2012-12-11 11:57:46,771::resourceManager::553::ResourceManager::(releaseResource) Released resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' (0 active users) >Thread-24::DEBUG::2012-12-11 11:57:46,771::resourceManager::558::ResourceManager::(releaseResource) Resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb' is free, finding out if anyone is waiting for it. >Thread-24::DEBUG::2012-12-11 11:57:46,771::resourceManager::565::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.d32a8696-910b-4222-a364-1c00b5303fbb', Clearing records. >Thread-24::ERROR::2012-12-11 11:57:46,771::dispatcher::66::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('d32a8696-910b-4222-a364-1c00b5303fbb',)", 'code': 309}} >Thread-25::DEBUG::2012-12-11 12:43:59,257::BindingXMLRPC::859::vds::(wrapper) client [10.35.148.46]::call getCapabilities with () {} >Thread-25::DEBUG::2012-12-11 12:43:59,277::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-25::DEBUG::2012-12-11 12:43:59,300::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-11 12:43:59,300::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-25::DEBUG::2012-12-11 12:43:59,316::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-11 12:43:59,317::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-25::DEBUG::2012-12-11 12:43:59,333::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-11 12:43:59,333::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-25::DEBUG::2012-12-11 12:43:59,349::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-11 12:43:59,349::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-25::DEBUG::2012-12-11 12:43:59,365::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-11 12:43:59,366::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '4.fc18.x86_64', 'buildtime': 1354623171.0, 'version': '3.6.9'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '1.fc18', 'buildtime': '1355100085', 'version': '0.10.2.2'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '10.35.18.155'}} >MainThread::INFO::2012-12-11 12:44:45,298::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::DEBUG::2012-12-11 12:44:45,299::task::588::TaskManager.Task::(_updateState) Task=`36f7c056-dbfe-497c-8ea8-15e3946b6a83`::moving from state init -> state preparing >MainThread::INFO::2012-12-11 12:44:45,299::logUtils::37::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-11::DEBUG::2012-12-11 12:44:45,300::storageServer::617::ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::DEBUG::2012-12-11 12:44:45,337::taskManager::80::TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2012-12-11 12:44:45,337::logUtils::39::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2012-12-11 12:44:45,337::task::1172::TaskManager.Task::(prepare) Task=`36f7c056-dbfe-497c-8ea8-15e3946b6a83`::finished: None >MainThread::DEBUG::2012-12-11 12:44:45,337::task::588::TaskManager.Task::(_updateState) Task=`36f7c056-dbfe-497c-8ea8-15e3946b6a83`::moving from state preparing -> state finished >MainThread::DEBUG::2012-12-11 12:44:45,337::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2012-12-11 12:44:45,338::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2012-12-11 12:44:45,338::task::978::TaskManager.Task::(_decref) Task=`36f7c056-dbfe-497c-8ea8-15e3946b6a83`::ref 0 aborting False >MainThread::INFO::2012-12-11 12:44:45,338::vdsm::75::vds::(run) VDSM main thread ended. Waiting for 14 other threads... >MainThread::INFO::2012-12-11 12:44:45,338::vdsm::78::vds::(run) <Thread(libvirtEventLoop, started daemon 139911196841728)> >MainThread::INFO::2012-12-11 12:44:45,338::vdsm::78::vds::(run) <WorkerThread(Thread-7, started daemon 139910707148544)> >MainThread::INFO::2012-12-11 12:44:45,338::vdsm::78::vds::(run) <WorkerThread(Thread-6, started daemon 139911057594112)> >MainThread::INFO::2012-12-11 12:44:45,338::vdsm::78::vds::(run) <WorkerThread(Thread-5, started daemon 139911065986816)> >MainThread::INFO::2012-12-11 12:44:45,338::vdsm::78::vds::(run) <WorkerThread(Thread-4, started daemon 139911074379520)> >MainThread::INFO::2012-12-11 12:44:45,339::vdsm::78::vds::(run) <WorkerThread(Thread-3, started daemon 139911082772224)> >MainThread::INFO::2012-12-11 12:44:45,339::vdsm::78::vds::(run) <Thread(Thread-22, started daemon 139910052812544)> >MainThread::INFO::2012-12-11 12:44:45,339::vdsm::78::vds::(run) <WorkerThread(Thread-10, started daemon 139910681970432)> >MainThread::INFO::2012-12-11 12:44:45,339::vdsm::78::vds::(run) <WorkerThread(Thread-9, started daemon 139910690363136)> >MainThread::INFO::2012-12-11 12:44:45,339::vdsm::78::vds::(run) <WorkerThread(Thread-8, started daemon 139910698755840)> >MainThread::INFO::2012-12-11 12:44:45,339::vdsm::78::vds::(run) <KsmMonitorThread(KsmMonitor, started daemon 139910103168768)> >MainThread::INFO::2012-12-11 12:44:45,339::vdsm::78::vds::(run) <_MainThread(MainThread, started 139911427282752)> >MainThread::INFO::2012-12-11 12:44:45,340::vdsm::78::vds::(run) <WorkerThread(Thread-2, started daemon 139911091164928)> >MainThread::INFO::2012-12-11 12:44:45,340::vmChannels::135::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2012-12-11 12:44:45,340::vdsm::78::vds::(run) <Listener(VM Channels Listener, started daemon 139910086383360)> >MainThread::INFO::2012-12-11 12:44:45,340::vdsm::78::vds::(run) <WorkerThread(Thread-1, started daemon 139911099557632)> >MainThread::INFO::2012-12-11 12:47:00,771::vdsm::70::vds::(run) I am the actual vdsm 4.10-11 >MainThread::DEBUG::2012-12-11 12:47:02,426::resourceManager::379::ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2012-12-11 12:47:02,427::threadPool::45::Misc.ThreadPool::(__init__) Enter - numThreads: 10.0, waitTimeout: 3, maxTasks: 500.0 >MainThread::DEBUG::2012-12-11 12:47:02,533::supervdsm::103::SuperVdsmProxy::(_killSupervdsm) Could not kill old Super Vdsm [Errno 2] No such file or directory: '/var/run/vdsm/svdsm.pid' >MainThread::DEBUG::2012-12-11 12:47:02,533::supervdsm::91::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm >MainThread::DEBUG::2012-12-11 12:47:02,533::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py a034a07a-239c-45b3-8ab7-a2f680b6c66e 1318' (cwd None) >MainThread::DEBUG::2012-12-11 12:47:02,673::supervdsmServer::306::SuperVdsm.Server::(main) Making sure I'm root >MainThread::DEBUG::2012-12-11 12:47:02,673::supervdsmServer::310::SuperVdsm.Server::(main) Parsing cmd args >MainThread::DEBUG::2012-12-11 12:47:02,673::supervdsmServer::313::SuperVdsm.Server::(main) Creating PID file >MainThread::DEBUG::2012-12-11 12:47:02,673::supervdsmServer::317::SuperVdsm.Server::(main) Cleaning old socket >MainThread::DEBUG::2012-12-11 12:47:02,673::supervdsmServer::321::SuperVdsm.Server::(main) Setting up keep alive thread >MainThread::DEBUG::2012-12-11 12:47:02,674::supervdsmServer::327::SuperVdsm.Server::(main) Creating remote object manager >MainThread::DEBUG::2012-12-11 12:47:02,675::supervdsmServer::338::SuperVdsm.Server::(main) Started serving super vdsm object >MainThread::DEBUG::2012-12-11 12:47:04,539::supervdsm::113::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2012-12-11 12:47:04,545::supervdsm::84::SuperVdsmProxy::(__init__) Connected to Super Vdsm >MainThread::DEBUG::2012-12-11 12:47:04,545::multipath::102::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2012-12-11 12:47:04,546::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2012-12-11 12:47:04,566::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 12:47:04,566::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 12:47:04,567::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%360a9800064775066476f6f3935524472|360a9800064775066476f6f3935526630|360a9800064775066476f6f3935533677%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::DEBUG::2012-12-11 12:47:04,596::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2012-12-11 12:47:04,596::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 12:47:04,597::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >MainThread::DEBUG::2012-12-11 12:47:04,597::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%360a9800064775066476f6f3935524472|360a9800064775066476f6f3935526630|360a9800064775066476f6f3935533677%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free' (cwd None) >MainThread::DEBUG::2012-12-11 12:47:04,623::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >MainThread::DEBUG::2012-12-11 12:47:04,624::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >MainThread::DEBUG::2012-12-11 12:47:04,624::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%360a9800064775066476f6f3935524472|360a9800064775066476f6f3935526630|360a9800064775066476f6f3935533677%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >MainThread::DEBUG::2012-12-11 12:47:04,650::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-12::DEBUG::2012-12-11 12:47:04,651::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2012-12-11 12:47:04,652::dispatcher::95::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >Thread-12::DEBUG::2012-12-11 12:47:04,652::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 12:47:04,652::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-12::DEBUG::2012-12-11 12:47:04,652::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-12::DEBUG::2012-12-11 12:47:04,653::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::DEBUG::2012-12-11 12:47:04,664::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/pgrep -xf ksmd' (cwd None) >Thread-12::DEBUG::2012-12-11 12:47:04,669::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 12:47:04,670::misc::1063::SamplingMethod::(__call__) Returning last result >MainThread::DEBUG::2012-12-11 12:47:04,677::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2012-12-11 12:47:04,678::vmChannels::139::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2012-12-11 12:47:04,678::vmChannels::127::vds::(run) Starting VM channels listener thread. >MainThread::ERROR::2012-12-11 12:47:04,681::clientIF::151::vds::(_prepareBindings) Unable to load the rest server module. Please make sure it is installed. >Thread-12::DEBUG::2012-12-11 12:47:06,504::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-12::DEBUG::2012-12-11 12:47:06,555::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-12::DEBUG::2012-12-11 12:47:06,555::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 12:47:06,555::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 12:47:06,556::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 12:47:06,556::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 12:47:06,556::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-12::DEBUG::2012-12-11 12:47:06,556::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-12::DEBUG::2012-12-11 12:47:06,556::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-12::DEBUG::2012-12-11 12:47:06,557::hsm::362::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >Thread-12::DEBUG::2012-12-11 12:47:06,560::hsm::394::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >Thread-12::DEBUG::2012-12-11 12:47:06,560::hsm::395::Storage.HSM::(__cleanStorageRepository) Mount list: [] >Thread-12::DEBUG::2012-12-11 12:47:06,560::hsm::397::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >Thread-12::DEBUG::2012-12-11 12:47:06,600::hsm::440::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >Thread-14::DEBUG::2012-12-11 12:47:19,337::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-14::DEBUG::2012-12-11 12:47:19,506::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-14::DEBUG::2012-12-11 12:47:19,522::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 12:47:19,523::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-14::DEBUG::2012-12-11 12:47:19,539::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 12:47:19,539::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-14::DEBUG::2012-12-11 12:47:19,555::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 12:47:19,555::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-14::DEBUG::2012-12-11 12:47:19,570::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 12:47:19,571::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-14::DEBUG::2012-12-11 12:47:19,586::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-14::DEBUG::2012-12-11 12:47:20,366::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '4.fc18.x86_64', 'buildtime': 1354623171.0, 'version': '3.6.9'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '1.fc18', 'buildtime': '1355100085', 'version': '0.10.2.2'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '0.0.0.0'}} >Thread-15::DEBUG::2012-12-11 12:47:20,629::BindingXMLRPC::859::vds::(wrapper) client [10.35.18.155]::call getCapabilities with () {} >Thread-15::DEBUG::2012-12-11 12:47:20,646::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) >Thread-15::DEBUG::2012-12-11 12:47:20,662::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 12:47:20,663::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) >Thread-15::DEBUG::2012-12-11 12:47:20,679::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 12:47:20,679::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) >Thread-15::DEBUG::2012-12-11 12:47:20,695::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 12:47:20,695::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) >Thread-15::DEBUG::2012-12-11 12:47:20,711::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 12:47:20,711::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) >Thread-15::DEBUG::2012-12-11 12:47:20,727::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2012-12-11 12:47:20,727::BindingXMLRPC::865::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2'}], 'FC': []}, 'packages2': {'kernel': {'release': '4.fc18.x86_64', 'buildtime': 1354623171.0, 'version': '3.6.9'}, 'spice-server': {'release': '1.fc18', 'buildtime': '1348825855', 'version': '0.12.0'}, 'vdsm': {'release': '11.fc18', 'buildtime': '1350749318', 'version': '4.10.0'}, 'qemu-kvm': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}, 'libvirt': {'release': '1.fc18', 'buildtime': '1355100085', 'version': '0.10.2.2'}, 'qemu-img': {'release': '24.fc18', 'buildtime': '1353584886', 'version': '1.2.0'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5620 @ 2.40GHz', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': '10.35.148.46', 'cfg': {'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'DEVICE': 'ovirtmgmt', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'bridged': True, 'gateway': '10.35.151.254', 'ports': ['em1']}}, 'uuid': '4C4C4544-0043-4610-805A-B7C04F37354A_00:26:6c:ff:c6:c4', 'lastClientIface': 'ovirtmgmt', 'nics': {'em1': {'hwaddr': '00:26:6c:ff:c6:c4', 'netmask': '', 'speed': 1000, 'addr': '', 'mtu': '1500'}, 'em2': {'hwaddr': '00:26:6c:ff:c6:c5', 'netmask': '', 'speed': 0, 'addr': '', 'mtu': '1500'}}, 'software_revision': '11', 'management_ip': '', 'clusterLevels': ['3.0', '3.1'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,ida,arat,epb,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_coreduo,model_Conroe', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:d4fad5fe4e2', 'netConfigDirty': 'False', 'memSize': '12007', 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.10', 'cpuSpeed': '2401.000', 'cpuSockets': '1', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0', '3.1'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-1.2', u'none', u'pc', u'pc-1.1', u'pc-1.0', u'pc-0.15', u'pc-0.14', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '0.8', 'version': '18', 'name': 'Fedora'}, 'lastClient': '10.35.18.155'}} >Thread-17::DEBUG::2012-12-11 12:47:23,090::task::588::TaskManager.Task::(_updateState) Task=`f2e3a018-3b42-4709-bcdf-980a7ecbf23b`::moving from state init -> state preparing >Thread-17::INFO::2012-12-11 12:47:23,090::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-17::INFO::2012-12-11 12:47:23,091::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-17::DEBUG::2012-12-11 12:47:23,091::task::1172::TaskManager.Task::(prepare) Task=`f2e3a018-3b42-4709-bcdf-980a7ecbf23b`::finished: {} >Thread-17::DEBUG::2012-12-11 12:47:23,091::task::588::TaskManager.Task::(_updateState) Task=`f2e3a018-3b42-4709-bcdf-980a7ecbf23b`::moving from state preparing -> state finished >Thread-17::DEBUG::2012-12-11 12:47:23,091::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-17::DEBUG::2012-12-11 12:47:23,091::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-17::DEBUG::2012-12-11 12:47:23,091::task::978::TaskManager.Task::(_decref) Task=`f2e3a018-3b42-4709-bcdf-980a7ecbf23b`::ref 0 aborting False >Thread-20::DEBUG::2012-12-11 12:47:26,350::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-20::DEBUG::2012-12-11 12:47:26,350::task::588::TaskManager.Task::(_updateState) Task=`a9d6554a-2667-4eb0-9299-45de382aa526`::moving from state init -> state preparing >Thread-20::INFO::2012-12-11 12:47:26,350::logUtils::37::dispatcher::(wrapper) Run and protect: discoverSendTargets(con={'connection': '10.35.148.12', 'password': '******', 'port': '3260', 'user': ''}, options=None) >Thread-20::DEBUG::2012-12-11 12:47:26,350::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.35.148.12:3260 --op=new' (cwd None) >Thread-20::DEBUG::2012-12-11 12:47:26,379::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2012-12-11 12:47:26,380::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.35.148.12:3260 -n node.startup -v manual --op=update' (cwd None) >Thread-20::DEBUG::2012-12-11 12:47:26,389::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: Cannot modify node.startup. Invalid param name.\n'; <rc> = 7 >Thread-20::DEBUG::2012-12-11 12:47:26,390::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m iface' (cwd None) >Thread-20::DEBUG::2012-12-11 12:47:26,413::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2012-12-11 12:47:26,414::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.35.148.12:3260 --discover' (cwd None) >Thread-20::DEBUG::2012-12-11 12:47:26,434::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::DEBUG::2012-12-11 12:47:26,435::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.35.148.12:3260 --op=delete' (cwd None) >Thread-20::DEBUG::2012-12-11 12:47:26,444::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::INFO::2012-12-11 12:47:26,445::logUtils::39::dispatcher::(wrapper) Run and protect: discoverSendTargets, Return response: {'fullTargets': ['10.35.148.12:3260,2000 iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e'], 'targets': ['iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e']} >Thread-20::DEBUG::2012-12-11 12:47:26,445::task::1172::TaskManager.Task::(prepare) Task=`a9d6554a-2667-4eb0-9299-45de382aa526`::finished: {'fullTargets': ['10.35.148.12:3260,2000 iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e'], 'targets': ['iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e']} >Thread-20::DEBUG::2012-12-11 12:47:26,445::task::588::TaskManager.Task::(_updateState) Task=`a9d6554a-2667-4eb0-9299-45de382aa526`::moving from state preparing -> state finished >Thread-20::DEBUG::2012-12-11 12:47:26,445::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-20::DEBUG::2012-12-11 12:47:26,445::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-20::DEBUG::2012-12-11 12:47:26,446::task::978::TaskManager.Task::(_decref) Task=`a9d6554a-2667-4eb0-9299-45de382aa526`::ref 0 aborting False >Thread-21::DEBUG::2012-12-11 12:47:27,239::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-21::DEBUG::2012-12-11 12:47:27,239::task::588::TaskManager.Task::(_updateState) Task=`837b23ae-2076-408c-a719-f199758e1ea6`::moving from state init -> state preparing >Thread-21::INFO::2012-12-11 12:47:27,240::logUtils::37::dispatcher::(wrapper) Run and protect: discoverSendTargets(con={'connection': '10.35.148.12', 'password': '******', 'port': '3260', 'user': ''}, options=None) >Thread-21::DEBUG::2012-12-11 12:47:27,240::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.35.148.12:3260 --op=new' (cwd None) >Thread-21::DEBUG::2012-12-11 12:47:27,250::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2012-12-11 12:47:27,250::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.35.148.12:3260 -n node.startup -v manual --op=update' (cwd None) >Thread-21::DEBUG::2012-12-11 12:47:27,260::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: Cannot modify node.startup. Invalid param name.\n'; <rc> = 7 >Thread-21::DEBUG::2012-12-11 12:47:27,260::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m iface' (cwd None) >Thread-21::DEBUG::2012-12-11 12:47:27,270::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2012-12-11 12:47:27,271::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.35.148.12:3260 --discover' (cwd None) >Thread-21::DEBUG::2012-12-11 12:47:29,785::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::DEBUG::2012-12-11 12:47:29,785::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.35.148.12:3260 --op=delete' (cwd None) >Thread-21::DEBUG::2012-12-11 12:47:29,795::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-21::INFO::2012-12-11 12:47:29,796::logUtils::39::dispatcher::(wrapper) Run and protect: discoverSendTargets, Return response: {'fullTargets': ['10.35.148.12:3260,2000 iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e'], 'targets': ['iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e']} >Thread-21::DEBUG::2012-12-11 12:47:29,796::task::1172::TaskManager.Task::(prepare) Task=`837b23ae-2076-408c-a719-f199758e1ea6`::finished: {'fullTargets': ['10.35.148.12:3260,2000 iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e'], 'targets': ['iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e']} >Thread-21::DEBUG::2012-12-11 12:47:29,796::task::588::TaskManager.Task::(_updateState) Task=`837b23ae-2076-408c-a719-f199758e1ea6`::moving from state preparing -> state finished >Thread-21::DEBUG::2012-12-11 12:47:29,796::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-21::DEBUG::2012-12-11 12:47:29,796::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-21::DEBUG::2012-12-11 12:47:29,796::task::978::TaskManager.Task::(_decref) Task=`837b23ae-2076-408c-a719-f199758e1ea6`::ref 0 aborting False >Thread-24::DEBUG::2012-12-11 12:47:30,421::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-24::DEBUG::2012-12-11 12:47:30,421::task::588::TaskManager.Task::(_updateState) Task=`1bdbd088-4a94-40b0-a826-180f5b4a3f73`::moving from state init -> state preparing >Thread-24::INFO::2012-12-11 12:47:30,422::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=3, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '0', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': '3260'}], options=None) >Thread-24::INFO::2012-12-11 12:47:30,422::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-24::DEBUG::2012-12-11 12:47:30,422::task::1172::TaskManager.Task::(prepare) Task=`1bdbd088-4a94-40b0-a826-180f5b4a3f73`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-24::DEBUG::2012-12-11 12:47:30,422::task::588::TaskManager.Task::(_updateState) Task=`1bdbd088-4a94-40b0-a826-180f5b4a3f73`::moving from state preparing -> state finished >Thread-24::DEBUG::2012-12-11 12:47:30,422::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-24::DEBUG::2012-12-11 12:47:30,422::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-24::DEBUG::2012-12-11 12:47:30,422::task::978::TaskManager.Task::(_decref) Task=`1bdbd088-4a94-40b0-a826-180f5b4a3f73`::ref 0 aborting False >Thread-25::DEBUG::2012-12-11 12:47:30,482::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-25::DEBUG::2012-12-11 12:47:30,482::task::588::TaskManager.Task::(_updateState) Task=`0c7ddf7c-2758-4d20-8782-9f4762b875d7`::moving from state init -> state preparing >Thread-25::INFO::2012-12-11 12:47:30,482::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '0', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': '3260'}], options=None) >Thread-25::DEBUG::2012-12-11 12:47:30,483::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 --op=new' (cwd None) >Thread-25::DEBUG::2012-12-11 12:47:30,493::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-11 12:47:30,493::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 -n node.startup -v manual --op=update' (cwd None) >Thread-25::DEBUG::2012-12-11 12:47:30,503::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-11 12:47:30,504::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 -l' (cwd None) >Thread-25::DEBUG::2012-12-11 12:47:30,514::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-25::DEBUG::2012-12-11 12:47:30,515::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-25::DEBUG::2012-12-11 12:47:30,515::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-25::DEBUG::2012-12-11 12:47:30,515::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-25::DEBUG::2012-12-11 12:47:30,515::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-25::DEBUG::2012-12-11 12:47:30,516::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-25::DEBUG::2012-12-11 12:47:30,516::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-25::INFO::2012-12-11 12:47:30,516::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-25::DEBUG::2012-12-11 12:47:30,516::task::1172::TaskManager.Task::(prepare) Task=`0c7ddf7c-2758-4d20-8782-9f4762b875d7`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-25::DEBUG::2012-12-11 12:47:30,516::task::588::TaskManager.Task::(_updateState) Task=`0c7ddf7c-2758-4d20-8782-9f4762b875d7`::moving from state preparing -> state finished >Thread-25::DEBUG::2012-12-11 12:47:30,516::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-25::DEBUG::2012-12-11 12:47:30,517::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-25::DEBUG::2012-12-11 12:47:30,517::task::978::TaskManager.Task::(_decref) Task=`0c7ddf7c-2758-4d20-8782-9f4762b875d7`::ref 0 aborting False >Thread-26::DEBUG::2012-12-11 12:47:31,015::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-26::DEBUG::2012-12-11 12:47:31,016::task::588::TaskManager.Task::(_updateState) Task=`0ee9b4ad-14a0-4cb3-bbc9-01bac073ae5c`::moving from state init -> state preparing >Thread-26::INFO::2012-12-11 12:47:31,016::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=3, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '0', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': '3260'}], options=None) >Thread-26::INFO::2012-12-11 12:47:31,016::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-26::DEBUG::2012-12-11 12:47:31,016::task::1172::TaskManager.Task::(prepare) Task=`0ee9b4ad-14a0-4cb3-bbc9-01bac073ae5c`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-26::DEBUG::2012-12-11 12:47:31,016::task::588::TaskManager.Task::(_updateState) Task=`0ee9b4ad-14a0-4cb3-bbc9-01bac073ae5c`::moving from state preparing -> state finished >Thread-26::DEBUG::2012-12-11 12:47:31,016::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-26::DEBUG::2012-12-11 12:47:31,017::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-26::DEBUG::2012-12-11 12:47:31,017::task::978::TaskManager.Task::(_decref) Task=`0ee9b4ad-14a0-4cb3-bbc9-01bac073ae5c`::ref 0 aborting False >Thread-27::DEBUG::2012-12-11 12:47:31,106::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-27::DEBUG::2012-12-11 12:47:31,106::task::588::TaskManager.Task::(_updateState) Task=`755b98e7-87a2-4996-b7ae-9ead81700933`::moving from state init -> state preparing >Thread-27::INFO::2012-12-11 12:47:31,107::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '0', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': '3260'}], options=None) >Thread-27::DEBUG::2012-12-11 12:47:31,107::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 --op=new' (cwd None) >Thread-27::DEBUG::2012-12-11 12:47:31,117::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-27::DEBUG::2012-12-11 12:47:31,118::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 -n node.startup -v manual --op=update' (cwd None) >Thread-27::DEBUG::2012-12-11 12:47:31,128::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-27::DEBUG::2012-12-11 12:47:31,128::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 -l' (cwd None) >Thread-27::DEBUG::2012-12-11 12:47:31,139::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-27::DEBUG::2012-12-11 12:47:31,139::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2012-12-11 12:47:31,139::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2012-12-11 12:47:31,140::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2012-12-11 12:47:31,140::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::DEBUG::2012-12-11 12:47:31,140::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-27::DEBUG::2012-12-11 12:47:31,140::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-27::INFO::2012-12-11 12:47:31,140::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-27::DEBUG::2012-12-11 12:47:31,141::task::1172::TaskManager.Task::(prepare) Task=`755b98e7-87a2-4996-b7ae-9ead81700933`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-27::DEBUG::2012-12-11 12:47:31,141::task::588::TaskManager.Task::(_updateState) Task=`755b98e7-87a2-4996-b7ae-9ead81700933`::moving from state preparing -> state finished >Thread-27::DEBUG::2012-12-11 12:47:31,141::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-27::DEBUG::2012-12-11 12:47:31,141::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-27::DEBUG::2012-12-11 12:47:31,141::task::978::TaskManager.Task::(_decref) Task=`755b98e7-87a2-4996-b7ae-9ead81700933`::ref 0 aborting False >Thread-29::DEBUG::2012-12-11 12:47:33,130::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-29::DEBUG::2012-12-11 12:47:33,130::task::588::TaskManager.Task::(_updateState) Task=`a4188999-5adb-4802-ba29-4a8354ed2b2b`::moving from state init -> state preparing >Thread-29::INFO::2012-12-11 12:47:33,131::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=3, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '0', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': '3260'}], options=None) >Thread-29::INFO::2012-12-11 12:47:33,131::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-29::DEBUG::2012-12-11 12:47:33,131::task::1172::TaskManager.Task::(prepare) Task=`a4188999-5adb-4802-ba29-4a8354ed2b2b`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-29::DEBUG::2012-12-11 12:47:33,131::task::588::TaskManager.Task::(_updateState) Task=`a4188999-5adb-4802-ba29-4a8354ed2b2b`::moving from state preparing -> state finished >Thread-29::DEBUG::2012-12-11 12:47:33,131::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-29::DEBUG::2012-12-11 12:47:33,131::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-29::DEBUG::2012-12-11 12:47:33,131::task::978::TaskManager.Task::(_decref) Task=`a4188999-5adb-4802-ba29-4a8354ed2b2b`::ref 0 aborting False >Thread-30::DEBUG::2012-12-11 12:47:33,200::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-30::DEBUG::2012-12-11 12:47:33,201::task::588::TaskManager.Task::(_updateState) Task=`4f6b0867-0f6a-4e38-adf7-a3749a511d9d`::moving from state init -> state preparing >Thread-30::INFO::2012-12-11 12:47:33,201::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '0', 'user': '', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000', 'port': '3260'}], options=None) >Thread-30::DEBUG::2012-12-11 12:47:33,201::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 --op=new' (cwd None) >Thread-30::DEBUG::2012-12-11 12:47:33,212::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-30::DEBUG::2012-12-11 12:47:33,212::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 -n node.startup -v manual --op=update' (cwd None) >Thread-30::DEBUG::2012-12-11 12:47:33,222::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-30::DEBUG::2012-12-11 12:47:33,222::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e -I default -p 10.35.148.12:3260 -l' (cwd None) >Thread-30::DEBUG::2012-12-11 12:47:33,233::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-30::DEBUG::2012-12-11 12:47:33,234::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-30::DEBUG::2012-12-11 12:47:33,234::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-30::DEBUG::2012-12-11 12:47:33,234::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-30::DEBUG::2012-12-11 12:47:33,234::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-30::DEBUG::2012-12-11 12:47:33,234::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-30::DEBUG::2012-12-11 12:47:33,234::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-30::INFO::2012-12-11 12:47:33,235::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-30::DEBUG::2012-12-11 12:47:33,235::task::1172::TaskManager.Task::(prepare) Task=`4f6b0867-0f6a-4e38-adf7-a3749a511d9d`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]} >Thread-30::DEBUG::2012-12-11 12:47:33,235::task::588::TaskManager.Task::(_updateState) Task=`4f6b0867-0f6a-4e38-adf7-a3749a511d9d`::moving from state preparing -> state finished >Thread-30::DEBUG::2012-12-11 12:47:33,235::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-30::DEBUG::2012-12-11 12:47:33,235::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-30::DEBUG::2012-12-11 12:47:33,235::task::978::TaskManager.Task::(_decref) Task=`4f6b0867-0f6a-4e38-adf7-a3749a511d9d`::ref 0 aborting False >Thread-31::DEBUG::2012-12-11 12:47:33,320::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-31::DEBUG::2012-12-11 12:47:33,320::task::588::TaskManager.Task::(_updateState) Task=`6778df13-eda9-49bd-baac-c8d1b89371c7`::moving from state init -> state preparing >Thread-31::INFO::2012-12-11 12:47:33,321::logUtils::37::dispatcher::(wrapper) Run and protect: getDeviceList(storageType=3, options={}) >Thread-31::DEBUG::2012-12-11 12:47:33,321::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-31::DEBUG::2012-12-11 12:47:33,321::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-31::DEBUG::2012-12-11 12:47:33,321::misc::1053::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-31::DEBUG::2012-12-11 12:47:33,321::misc::1055::SamplingMethod::(__call__) Got in to sampling method >Thread-31::DEBUG::2012-12-11 12:47:33,321::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-31::DEBUG::2012-12-11 12:47:33,338::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-31::DEBUG::2012-12-11 12:47:33,338::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-32::DEBUG::2012-12-11 12:47:33,744::task::588::TaskManager.Task::(_updateState) Task=`134b94fb-60d3-45a2-9f57-0fffbeed3f6c`::moving from state init -> state preparing >Thread-32::INFO::2012-12-11 12:47:33,744::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-32::INFO::2012-12-11 12:47:33,744::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-32::DEBUG::2012-12-11 12:47:33,744::task::1172::TaskManager.Task::(prepare) Task=`134b94fb-60d3-45a2-9f57-0fffbeed3f6c`::finished: {} >Thread-32::DEBUG::2012-12-11 12:47:33,744::task::588::TaskManager.Task::(_updateState) Task=`134b94fb-60d3-45a2-9f57-0fffbeed3f6c`::moving from state preparing -> state finished >Thread-32::DEBUG::2012-12-11 12:47:33,745::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-32::DEBUG::2012-12-11 12:47:33,745::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-32::DEBUG::2012-12-11 12:47:33,745::task::978::TaskManager.Task::(_decref) Task=`134b94fb-60d3-45a2-9f57-0fffbeed3f6c`::ref 0 aborting False >Thread-31::DEBUG::2012-12-11 12:47:35,173::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/multipath' (cwd None) >Thread-31::DEBUG::2012-12-11 12:47:35,220::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-31::DEBUG::2012-12-11 12:47:35,220::lvm::457::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-31::DEBUG::2012-12-11 12:47:35,221::lvm::459::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-31::DEBUG::2012-12-11 12:47:35,221::lvm::469::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-31::DEBUG::2012-12-11 12:47:35,221::lvm::471::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-31::DEBUG::2012-12-11 12:47:35,221::lvm::490::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-31::DEBUG::2012-12-11 12:47:35,221::lvm::492::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-31::DEBUG::2012-12-11 12:47:35,221::misc::1063::SamplingMethod::(__call__) Returning last result >Thread-31::DEBUG::2012-12-11 12:47:35,222::lvm::316::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-31::DEBUG::2012-12-11 12:47:35,223::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%360a9800064775066476f6f3935524472|360a9800064775066476f6f3935526630|360a9800064775066476f6f3935533677%\\", \\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >Thread-31::DEBUG::2012-12-11 12:47:35,256::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-31::DEBUG::2012-12-11 12:47:35,256::lvm::339::OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >MainProcess|Thread-31::DEBUG::2012-12-11 12:47:35,257::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/sbin/dmsetup status' (cwd None) >MainProcess|Thread-31::DEBUG::2012-12-11 12:47:35,259::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainProcess|Thread-31::DEBUG::2012-12-11 12:47:35,261::__init__::1249::Storage.Misc.excCmd::(_log) '/lib/udev/scsi_id --page=0x80 --whitelisted --export --replace-whitespace --device=/dev/dm-0' (cwd None) >MainProcess|Thread-31::DEBUG::2012-12-11 12:47:35,264::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainProcess|Thread-31::DEBUG::2012-12-11 12:47:35,267::__init__::1249::Storage.Misc.excCmd::(_log) '/lib/udev/scsi_id --page=0x80 --whitelisted --export --replace-whitespace --device=/dev/dm-1' (cwd None) >MainProcess|Thread-31::DEBUG::2012-12-11 12:47:35,269::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >MainProcess|Thread-31::DEBUG::2012-12-11 12:47:35,272::__init__::1249::Storage.Misc.excCmd::(_log) '/lib/udev/scsi_id --page=0x80 --whitelisted --export --replace-whitespace --device=/dev/dm-2' (cwd None) >MainProcess|Thread-31::DEBUG::2012-12-11 12:47:35,274::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0 >Thread-31::INFO::2012-12-11 12:47:35,277::logUtils::39::dispatcher::(wrapper) Run and protect: getDeviceList, Return response: {'devList': [{'vendorID': 'NETAPP', 'capacity': '53687091200', 'fwrev': '8020', 'partitioned': False, 'vgUUID': '', 'pathlist': [{'initiatorname': 'default', 'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '2000', 'password': '******', 'port': '3260'}], 'logicalblocksize': '512', 'pathstatus': [{'physdev': 'sdd', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': '', 'serial': 'SNETAPP_LUN_dwPfGoo95S6w', 'GUID': '360a9800064775066476f6f3935533677', 'productID': 'LUN'}, {'vendorID': 'NETAPP', 'capacity': '53687091200', 'fwrev': '8020', 'partitioned': False, 'vgUUID': '', 'pathlist': [{'initiatorname': 'default', 'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '2000', 'password': '******', 'port': '3260'}], 'logicalblocksize': '512', 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': '', 'serial': 'SNETAPP_LUN_dwPfGoo95Rf0', 'GUID': '360a9800064775066476f6f3935526630', 'productID': 'LUN'}, {'vendorID': 'NETAPP', 'capacity': '53687091200', 'fwrev': '8020', 'partitioned': False, 'vgUUID': '', 'pathlist': [{'initiatorname': 'default', 'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '2000', 'password': '******', 'port': '3260'}], 'logicalblocksize': '512', 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '0'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': '', 'serial': 'SNETAPP_LUN_dwPfGoo95RDr', 'GUID': '360a9800064775066476f6f3935524472', 'productID': 'LUN'}]} >Thread-31::DEBUG::2012-12-11 12:47:35,277::task::1172::TaskManager.Task::(prepare) Task=`6778df13-eda9-49bd-baac-c8d1b89371c7`::finished: {'devList': [{'fwrev': '8020', 'vgUUID': '', 'pathlist': [{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '2000', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SNETAPP_LUN_dwPfGoo95S6w', 'GUID': '360a9800064775066476f6f3935533677', 'productID': 'LUN', 'vendorID': 'NETAPP', 'capacity': '53687091200', 'partitioned': False, 'pathstatus': [{'physdev': 'sdd', 'state': 'active', 'type': 'iSCSI', 'lun': '2'}], 'pvUUID': ''}, {'fwrev': '8020', 'vgUUID': '', 'pathlist': [{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '2000', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SNETAPP_LUN_dwPfGoo95Rf0', 'GUID': '360a9800064775066476f6f3935526630', 'productID': 'LUN', 'vendorID': 'NETAPP', 'capacity': '53687091200', 'partitioned': False, 'pathstatus': [{'physdev': 'sdc', 'state': 'active', 'type': 'iSCSI', 'lun': '1'}], 'pvUUID': ''}, {'fwrev': '8020', 'vgUUID': '', 'pathlist': [{'connection': '10.35.148.12', 'iqn': 'iqn.1992-08.com.netapp:sn.1574937817:vf.af72f996-a4d9-11e1-a00b-00a0981c197e', 'portal': '2000', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SNETAPP_LUN_dwPfGoo95RDr', 'GUID': '360a9800064775066476f6f3935524472', 'productID': 'LUN', 'vendorID': 'NETAPP', 'capacity': '53687091200', 'partitioned': False, 'pathstatus': [{'physdev': 'sdb', 'state': 'active', 'type': 'iSCSI', 'lun': '0'}], 'pvUUID': ''}]} >Thread-31::DEBUG::2012-12-11 12:47:35,277::task::588::TaskManager.Task::(_updateState) Task=`6778df13-eda9-49bd-baac-c8d1b89371c7`::moving from state preparing -> state finished >Thread-31::DEBUG::2012-12-11 12:47:35,277::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-31::DEBUG::2012-12-11 12:47:35,278::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-31::DEBUG::2012-12-11 12:47:35,278::task::978::TaskManager.Task::(_decref) Task=`6778df13-eda9-49bd-baac-c8d1b89371c7`::ref 0 aborting False >Thread-34::DEBUG::2012-12-11 12:47:35,546::BindingXMLRPC::156::vds::(wrapper) [10.35.18.155] >Thread-34::ERROR::2012-12-11 12:47:35,546::BindingXMLRPC::160::vds::(wrapper) Unexpected exception >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 158, in wrapper > return f(*args, **kwargs) >TypeError: vgCreate() takes exactly 3 arguments (4 given) >Thread-39::DEBUG::2012-12-11 12:47:44,081::task::588::TaskManager.Task::(_updateState) Task=`538704a3-707b-440e-86b8-8793f876555b`::moving from state init -> state preparing >Thread-39::INFO::2012-12-11 12:47:44,081::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-39::INFO::2012-12-11 12:47:44,081::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-39::DEBUG::2012-12-11 12:47:44,081::task::1172::TaskManager.Task::(prepare) Task=`538704a3-707b-440e-86b8-8793f876555b`::finished: {} >Thread-39::DEBUG::2012-12-11 12:47:44,081::task::588::TaskManager.Task::(_updateState) Task=`538704a3-707b-440e-86b8-8793f876555b`::moving from state preparing -> state finished >Thread-39::DEBUG::2012-12-11 12:47:44,082::resourceManager::809::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-39::DEBUG::2012-12-11 12:47:44,082::resourceManager::844::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-39::DEBUG::2012-12-11 12:47:44,082::task::978::TaskManager.Task::(_decref) Task=`538704a3-707b-440e-86b8-8793f876555b`::ref 0 aborting False
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 886087
: 669598 |
669599