Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 854405 Details for
Bug 1057139
[oVirt] fail to start VM due to FirewallD not running
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
VDSM log
vdsm.log (text/x-log), 269.32 KB, created by
Mike Kolesnik
on 2014-01-23 14:03:44 UTC
(
hide
)
Description:
VDSM log
Filename:
MIME Type:
Creator:
Mike Kolesnik
Created:
2014-01-23 14:03:44 UTC
Size:
269.32 KB
patch
obsolete
>MainThread::INFO::2014-01-23 15:01:14,501::vdsm::119::vds::(run) (PID: 1151) I am the actual vdsm 4.14.1-2.fc19 localhost.localdomain (3.11.3-201.fc19.x86_64) >MainThread::DEBUG::2014-01-23 15:01:14,652::multipath::151::Storage.Multipath::(isEnabled) multipath Defaulting to False >MainThread::DEBUG::2014-01-23 15:01:14,653::multipath::169::Storage.Misc.excCmd::(setupMultipath) '/usr/bin/sudo -n /usr/bin/cp /tmp/tmpeNGnHa /etc/multipath.conf' (cwd None) >MainThread::DEBUG::2014-01-23 15:01:14,662::multipath::169::Storage.Misc.excCmd::(setupMultipath) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:01:14,663::multipath::175::Storage.Misc.excCmd::(setupMultipath) '/usr/bin/sudo -n /sbin/multipath -F' (cwd None) >MainThread::DEBUG::2014-01-23 15:01:14,694::multipath::175::Storage.Misc.excCmd::(setupMultipath) FAILED: <err> = ''; <rc> = 1 >MainThread::DEBUG::2014-01-23 15:01:14,694::multipath::178::Storage.Misc.excCmd::(setupMultipath) '/usr/bin/sudo -n /usr/bin/vdsm-tool service-reload multipathd' (cwd None) >MainThread::DEBUG::2014-01-23 15:01:14,895::multipath::178::Storage.Misc.excCmd::(setupMultipath) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:01:14,895::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2014-01-23 15:01:14,915::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:01:14,915::hsm::447::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-01-23 15:01:14,918::hsm::479::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-01-23 15:01:14,918::hsm::480::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-01-23 15:01:14,918::hsm::482::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-01-23 15:01:14,919::hsm::525::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2014-01-23 15:01:14,920::dispatcher::99::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-01-23 15:01:14,924::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-01-23 15:01:14,931::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::INFO::2014-01-23 15:01:15,000::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x25e7f38>>) >storageRefresh::DEBUG::2014-01-23 15:01:15,000::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >MainThread::INFO::2014-01-23 15:01:15,001::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::INFO::2014-01-23 15:01:15,008::momIF::47::MOM::(__init__) Starting up MOM >storageRefresh::DEBUG::2014-01-23 15:01:15,012::multipath::110::Storage.Misc.excCmd::(rescan) FAILED: <err> = ''; <rc> = 1 >storageRefresh::DEBUG::2014-01-23 15:01:15,017::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::INFO::2014-01-23 15:01:15,020::vmChannels::187::vds::(settimeout) Setting channels' timeout to 30 seconds. >clientIFinit::DEBUG::2014-01-23 15:01:15,044::libvirtconnection::145::root::(get) trying to connect libvirt >MainThread::WARNING::2014-01-23 15:01:15,136::clientIF::181::vds::(_prepareBindings) Unable to load the json rpc server module. Please make sure it is installed. >storageRefresh::DEBUG::2014-01-23 15:01:15,138::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:01:15,139::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >VM Channels Listener::INFO::2014-01-23 15:01:15,047::vmChannels::170::vds::(run) Starting VM channels listener thread. >storageRefresh::DEBUG::2014-01-23 15:01:15,189::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:01:15,190::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:01:15,227::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:01:15,233::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-13::DEBUG::2014-01-23 15:01:15,260::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-13::DEBUG::2014-01-23 15:01:15,261::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Recovering from crash or Initializing', 'code': 99}} >storageRefresh::DEBUG::2014-01-23 15:01:15,262::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:01:15,262::hsm::391::Storage.HSM::(storageRefresh) HSM is ready >Thread-14::DEBUG::2014-01-23 15:01:16,276::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-14::DEBUG::2014-01-23 15:01:16,320::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-15::DEBUG::2014-01-23 15:01:16,396::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-15::INFO::2014-01-23 15:01:16,397::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-15::INFO::2014-01-23 15:01:16,397::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-16::DEBUG::2014-01-23 15:02:47,068::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-16::INFO::2014-01-23 15:02:47,068::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '2479abe4-840d-4006-b7b9-dcdd4ecd0f61'}], options=None) >Thread-16::DEBUG::2014-01-23 15:02:47,071::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-16::DEBUG::2014-01-23 15:02:47,208::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-16::DEBUG::2014-01-23 15:02:47,210::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-16::DEBUG::2014-01-23 15:02:47,211::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-16::INFO::2014-01-23 15:02:47,211::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '2479abe4-840d-4006-b7b9-dcdd4ecd0f61'}]} >Thread-17::DEBUG::2014-01-23 15:02:47,223::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-17::INFO::2014-01-23 15:02:47,224::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=None, storageType=None, remotePath=None, options=None) >Thread-17::DEBUG::2014-01-23 15:02:47,224::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-17::DEBUG::2014-01-23 15:02:47,224::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-17::DEBUG::2014-01-23 15:02:47,235::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-17::DEBUG::2014-01-23 15:02:47,247::multipath::110::Storage.Misc.excCmd::(rescan) FAILED: <err> = ''; <rc> = 1 >Thread-17::DEBUG::2014-01-23 15:02:47,249::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-17::DEBUG::2014-01-23 15:02:47,272::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-17::INFO::2014-01-23 15:02:47,373::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-19::DEBUG::2014-01-23 15:02:47,385::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-19::INFO::2014-01-23 15:02:47,386::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '2479abe4-840d-4006-b7b9-dcdd4ecd0f61'}], options=None) >Thread-19::DEBUG::2014-01-23 15:02:47,386::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-19::DEBUG::2014-01-23 15:02:47,408::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-19::DEBUG::2014-01-23 15:02:47,409::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-19::DEBUG::2014-01-23 15:02:47,419::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-19::DEBUG::2014-01-23 15:02:47,431::multipath::110::Storage.Misc.excCmd::(rescan) FAILED: <err> = ''; <rc> = 1 >Thread-19::INFO::2014-01-23 15:02:47,432::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '2479abe4-840d-4006-b7b9-dcdd4ecd0f61'}]} >Thread-20::DEBUG::2014-01-23 15:04:06,267::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-20::DEBUG::2014-01-23 15:04:06,269::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-21::DEBUG::2014-01-23 15:04:06,339::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-21::INFO::2014-01-23 15:04:06,340::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-21::INFO::2014-01-23 15:04:06,340::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-22::DEBUG::2014-01-23 15:04:11,283::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-22::INFO::2014-01-23 15:04:11,284::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '012aaa01-82ff-4a6c-af48-c5c66244c5d1'}], options=None) >Thread-22::DEBUG::2014-01-23 15:04:11,287::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-22::DEBUG::2014-01-23 15:04:11,340::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-22::DEBUG::2014-01-23 15:04:11,344::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-22::DEBUG::2014-01-23 15:04:11,344::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-22::INFO::2014-01-23 15:04:11,344::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '012aaa01-82ff-4a6c-af48-c5c66244c5d1'}]} >Thread-23::DEBUG::2014-01-23 15:04:11,356::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-23::INFO::2014-01-23 15:04:11,356::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=None, storageType=None, remotePath=None, options=None) >Thread-23::DEBUG::2014-01-23 15:04:11,357::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-23::DEBUG::2014-01-23 15:04:11,357::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-23::DEBUG::2014-01-23 15:04:11,368::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-23::DEBUG::2014-01-23 15:04:11,382::multipath::110::Storage.Misc.excCmd::(rescan) FAILED: <err> = ''; <rc> = 1 >Thread-23::DEBUG::2014-01-23 15:04:11,391::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-23::DEBUG::2014-01-23 15:04:11,414::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-23::INFO::2014-01-23 15:04:11,416::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-25::DEBUG::2014-01-23 15:04:11,428::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-25::INFO::2014-01-23 15:04:11,428::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '012aaa01-82ff-4a6c-af48-c5c66244c5d1'}], options=None) >Thread-25::DEBUG::2014-01-23 15:04:11,428::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-25::DEBUG::2014-01-23 15:04:11,452::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-25::DEBUG::2014-01-23 15:04:11,453::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-25::DEBUG::2014-01-23 15:04:11,463::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-25::DEBUG::2014-01-23 15:04:11,475::multipath::110::Storage.Misc.excCmd::(rescan) FAILED: <err> = ''; <rc> = 1 >Thread-25::INFO::2014-01-23 15:04:11,476::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '012aaa01-82ff-4a6c-af48-c5c66244c5d1'}]} >Thread-26::DEBUG::2014-01-23 15:04:32,515::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-26::DEBUG::2014-01-23 15:04:32,518::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-27::DEBUG::2014-01-23 15:04:32,590::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-27::INFO::2014-01-23 15:04:32,591::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-27::INFO::2014-01-23 15:04:32,591::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-28::DEBUG::2014-01-23 15:04:35,007::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-28::INFO::2014-01-23 15:04:35,008::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': 'a6247807-c1bf-47e0-92d8-e61898c9ba2a'}], options=None) >Thread-28::DEBUG::2014-01-23 15:04:35,011::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-28::DEBUG::2014-01-23 15:04:35,061::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-28::DEBUG::2014-01-23 15:04:35,065::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-28::DEBUG::2014-01-23 15:04:35,065::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-28::INFO::2014-01-23 15:04:35,065::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'a6247807-c1bf-47e0-92d8-e61898c9ba2a'}]} >Thread-29::DEBUG::2014-01-23 15:04:35,075::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-29::INFO::2014-01-23 15:04:35,076::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=None, storageType=None, remotePath=None, options=None) >Thread-29::DEBUG::2014-01-23 15:04:35,076::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-29::DEBUG::2014-01-23 15:04:35,076::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-29::DEBUG::2014-01-23 15:04:35,088::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-29::DEBUG::2014-01-23 15:04:35,100::multipath::110::Storage.Misc.excCmd::(rescan) FAILED: <err> = ''; <rc> = 1 >Thread-29::DEBUG::2014-01-23 15:04:35,102::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-29::DEBUG::2014-01-23 15:04:35,124::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-29::INFO::2014-01-23 15:04:35,148::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-31::DEBUG::2014-01-23 15:04:35,161::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-31::INFO::2014-01-23 15:04:35,161::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': 'a6247807-c1bf-47e0-92d8-e61898c9ba2a'}], options=None) >Thread-31::DEBUG::2014-01-23 15:04:35,161::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-31::DEBUG::2014-01-23 15:04:35,182::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-31::DEBUG::2014-01-23 15:04:35,182::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-31::DEBUG::2014-01-23 15:04:35,192::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-31::DEBUG::2014-01-23 15:04:35,205::multipath::110::Storage.Misc.excCmd::(rescan) FAILED: <err> = ''; <rc> = 1 >Thread-31::INFO::2014-01-23 15:04:35,249::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'a6247807-c1bf-47e0-92d8-e61898c9ba2a'}]} >MainThread::DEBUG::2014-01-23 15:06:00,378::vdsm::55::vds::(sigtermHandler) Received signal 15 >MainThread::INFO::2014-01-23 15:06:01,258::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:06:01,258::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:06:01,259::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >MainThread::INFO::2014-01-23 15:06:01,360::domainMonitor::125::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-01-23 15:06:01,360::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::INFO::2014-01-23 15:06:01,361::vdsm::125::vds::(run) VDSM main thread ended. Waiting for 16 other threads... >MainThread::INFO::2014-01-23 15:06:01,361::vdsm::129::vds::(run) <Thread(libvirtEventLoop, started daemon 139820805797632)> >MainThread::INFO::2014-01-23 15:06:01,361::vdsm::129::vds::(run) <WorkerThread(Thread-9, started daemon 139820110182144)> >MainThread::INFO::2014-01-23 15:06:01,361::vdsm::129::vds::(run) <WorkerThread(Thread-2, started daemon 139820714161920)> >MainThread::INFO::2014-01-23 15:06:01,361::vdsm::129::vds::(run) <WorkerThread(Thread-8, started daemon 139820663805696)> >MainThread::INFO::2014-01-23 15:06:01,361::vdsm::129::vds::(run) <PolicyEngine(PolicyEngine, started daemon 139820059825920)> >MainThread::INFO::2014-01-23 15:06:01,361::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <MomThread(MOM, started 139820068218624)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <WorkerThread(Thread-1, started daemon 139820797404928)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <WorkerThread(Thread-3, started daemon 139820705769216)> >MainThread::INFO::2014-01-23 15:06:01,362::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <Listener(VM Channels Listener, started daemon 139819707528960)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <GuestManager(GuestManager, started daemon 139819675948800)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <WorkerThread(Thread-6, started daemon 139820680591104)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <_MainThread(MainThread, started 139821017823040)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <WorkerThread(Thread-5, started daemon 139820688983808)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <HostMonitor(HostMonitor, started daemon 139819690743552)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <WorkerThread(Thread-7, started daemon 139820672198400)> >MainThread::INFO::2014-01-23 15:06:01,362::vdsm::129::vds::(run) <WorkerThread(Thread-10, started daemon 139820101789440)> >MainThread::INFO::2014-01-23 15:06:01,363::vdsm::129::vds::(run) <WorkerThread(Thread-4, started daemon 139820697376512)> >VM Channels Listener::INFO::2014-01-23 15:06:01,459::vmChannels::178::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-01-23 15:10:15,687::vdsm::119::vds::(run) (PID: 3023) I am the actual vdsm 4.14.1-2.fc19 localhost.localdomain (3.12.8-200.fc19.x86_64) >MainThread::WARNING::2014-01-23 15:10:15,693::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-01-23 15:10:15,844::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-01-23 15:10:15,844::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2014-01-23 15:10:15,862::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:10:15,863::hsm::447::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-01-23 15:10:15,865::hsm::479::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-01-23 15:10:15,865::hsm::480::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-01-23 15:10:15,866::hsm::482::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-01-23 15:10:15,866::hsm::525::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2014-01-23 15:10:15,868::dispatcher::99::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-01-23 15:10:15,868::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-01-23 15:10:15,937::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:10:16,008::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >MainThread::INFO::2014-01-23 15:10:16,009::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x1704cb0>>) >MainThread::INFO::2014-01-23 15:10:16,012::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::INFO::2014-01-23 15:10:16,014::momIF::47::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-01-23 15:10:16,016::vmChannels::187::vds::(settimeout) Setting channels' timeout to 30 seconds. >storageRefresh::DEBUG::2014-01-23 15:10:16,063::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >clientIFinit::DEBUG::2014-01-23 15:10:16,063::libvirtconnection::145::root::(get) trying to connect libvirt >VM Channels Listener::INFO::2014-01-23 15:10:16,064::vmChannels::170::vds::(run) Starting VM channels listener thread. >storageRefresh::DEBUG::2014-01-23 15:10:16,070::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::WARNING::2014-01-23 15:10:16,071::clientIF::181::vds::(_prepareBindings) Unable to load the json rpc server module. Please make sure it is installed. >storageRefresh::DEBUG::2014-01-23 15:10:16,171::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:10:16,177::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:10:16,228::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:10:16,256::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:10:16,279::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:10:16,283::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:10:16,301::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::WARNING::2014-01-23 15:10:16,304::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/hsm-tasks already exists >storageRefresh::DEBUG::2014-01-23 15:10:16,306::hsm::391::Storage.HSM::(storageRefresh) HSM is ready >Thread-13::DEBUG::2014-01-23 15:10:32,067::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-13::DEBUG::2014-01-23 15:10:32,114::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-14::DEBUG::2014-01-23 15:10:32,184::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-14::INFO::2014-01-23 15:10:32,185::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-14::INFO::2014-01-23 15:10:32,185::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-15::DEBUG::2014-01-23 15:13:30,676::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-15::INFO::2014-01-23 15:13:30,677::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': 'af7abf0b-c901-4a24-983c-b1ec4d34311d'}], options=None) >Thread-15::DEBUG::2014-01-23 15:13:30,686::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-15::DEBUG::2014-01-23 15:13:30,836::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-15::DEBUG::2014-01-23 15:13:30,839::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-15::DEBUG::2014-01-23 15:13:30,839::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-15::INFO::2014-01-23 15:13:30,839::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'af7abf0b-c901-4a24-983c-b1ec4d34311d'}]} >Thread-16::DEBUG::2014-01-23 15:13:30,852::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-16::INFO::2014-01-23 15:13:30,852::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=None, storageType=None, remotePath=None, options=None) >Thread-16::DEBUG::2014-01-23 15:13:30,853::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-16::DEBUG::2014-01-23 15:13:30,853::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-16::DEBUG::2014-01-23 15:13:30,865::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-16::DEBUG::2014-01-23 15:13:30,894::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-01-23 15:13:30,896::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-16::DEBUG::2014-01-23 15:13:30,924::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-16::INFO::2014-01-23 15:13:31,023::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-18::DEBUG::2014-01-23 15:13:31,035::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-18::INFO::2014-01-23 15:13:31,035::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': 'af7abf0b-c901-4a24-983c-b1ec4d34311d'}], options=None) >Thread-18::DEBUG::2014-01-23 15:13:31,036::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-18::DEBUG::2014-01-23 15:13:31,055::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-18::DEBUG::2014-01-23 15:13:31,055::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-18::DEBUG::2014-01-23 15:13:31,065::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-18::DEBUG::2014-01-23 15:13:31,176::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::INFO::2014-01-23 15:13:31,177::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': 'af7abf0b-c901-4a24-983c-b1ec4d34311d'}]} >MainThread::DEBUG::2014-01-23 15:13:41,595::vdsm::55::vds::(sigtermHandler) Received signal 15 >MainThread::INFO::2014-01-23 15:13:42,047::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:13:42,048::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:13:42,049::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >MainThread::INFO::2014-01-23 15:13:42,188::domainMonitor::125::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-01-23 15:13:42,188::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::INFO::2014-01-23 15:13:42,189::vdsm::125::vds::(run) VDSM main thread ended. Waiting for 16 other threads... >MainThread::INFO::2014-01-23 15:13:42,189::vdsm::129::vds::(run) <Thread(libvirtEventLoop, started daemon 140676772566784)> >MainThread::INFO::2014-01-23 15:13:42,189::vdsm::129::vds::(run) <WorkerThread(Thread-7, started daemon 140676645758720)> >MainThread::INFO::2014-01-23 15:13:42,189::vdsm::129::vds::(run) <WorkerThread(Thread-2, started daemon 140676687722240)> >MainThread::INFO::2014-01-23 15:13:42,189::vdsm::129::vds::(run) <WorkerThread(Thread-8, started daemon 140676637366016)> >MainThread::INFO::2014-01-23 15:13:42,189::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:13:42,189::vdsm::129::vds::(run) <MomThread(MOM, started 140676041778944)> >MainThread::INFO::2014-01-23 15:13:42,189::vdsm::129::vds::(run) <WorkerThread(Thread-5, started daemon 140676662544128)> >MainThread::INFO::2014-01-23 15:13:42,189::vdsm::129::vds::(run) <WorkerThread(Thread-10, started daemon 140676075349760)> >MainThread::INFO::2014-01-23 15:13:42,190::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <Listener(VM Channels Listener, started daemon 140675681089280)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <WorkerThread(Thread-4, started daemon 140676670936832)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <WorkerThread(Thread-6, started daemon 140676654151424)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <_MainThread(MainThread, started 140676984592192)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140676058564352)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <WorkerThread(Thread-3, started daemon 140676679329536)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <WorkerThread(Thread-9, started daemon 140676083742464)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <WorkerThread(Thread-1, started daemon 140676764174080)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <GuestManager(GuestManager, started daemon 140675641022208)> >MainThread::INFO::2014-01-23 15:13:42,190::vdsm::129::vds::(run) <HostMonitor(HostMonitor, started daemon 140675672696576)> >VM Channels Listener::INFO::2014-01-23 15:13:42,261::vmChannels::178::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-01-23 15:13:51,900::vdsm::119::vds::(run) (PID: 3577) I am the actual vdsm 4.14.1-2.fc19 localhost.localdomain (3.12.8-200.fc19.x86_64) >MainThread::WARNING::2014-01-23 15:13:51,903::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-01-23 15:13:52,044::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-01-23 15:13:52,044::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2014-01-23 15:13:52,069::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:13:52,069::hsm::447::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-01-23 15:13:52,072::hsm::479::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-01-23 15:13:52,072::hsm::480::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-01-23 15:13:52,072::hsm::482::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-01-23 15:13:52,073::hsm::525::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >MainThread::INFO::2014-01-23 15:13:52,074::dispatcher::99::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-01-23 15:13:52,074::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-01-23 15:13:52,083::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::INFO::2014-01-23 15:13:52,117::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x1d3fd40>>) >MainThread::INFO::2014-01-23 15:13:52,117::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::INFO::2014-01-23 15:13:52,119::momIF::47::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-01-23 15:13:52,123::vmChannels::187::vds::(settimeout) Setting channels' timeout to 30 seconds. >clientIFinit::DEBUG::2014-01-23 15:13:52,140::libvirtconnection::145::root::(get) trying to connect libvirt >MainThread::WARNING::2014-01-23 15:13:52,145::clientIF::181::vds::(_prepareBindings) Unable to load the json rpc server module. Please make sure it is installed. >VM Channels Listener::INFO::2014-01-23 15:13:52,140::vmChannels::170::vds::(run) Starting VM channels listener thread. >Thread-13::DEBUG::2014-01-23 15:13:52,758::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-13::DEBUG::2014-01-23 15:13:52,758::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Recovering from crash or Initializing', 'code': 99}} >Thread-14::DEBUG::2014-01-23 15:13:53,769::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-14::DEBUG::2014-01-23 15:13:53,769::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Recovering from crash or Initializing', 'code': 99}} >storageRefresh::DEBUG::2014-01-23 15:13:54,098::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:13:54,135::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:13:54,137::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:13:54,165::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:13:54,198::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:13:54,214::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:13:54,215::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:13:54,236::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:13:54,237::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:13:54,252::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::WARNING::2014-01-23 15:13:54,253::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/hsm-tasks already exists >storageRefresh::DEBUG::2014-01-23 15:13:54,253::hsm::391::Storage.HSM::(storageRefresh) HSM is ready >Thread-15::DEBUG::2014-01-23 15:13:54,781::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-15::DEBUG::2014-01-23 15:13:54,801::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-16::DEBUG::2014-01-23 15:13:54,886::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-16::INFO::2014-01-23 15:13:54,887::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-16::INFO::2014-01-23 15:13:54,887::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >MainThread::DEBUG::2014-01-23 15:14:14,189::vdsm::55::vds::(sigtermHandler) Received signal 15 >MainThread::INFO::2014-01-23 15:14:15,910::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:14:15,911::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:14:15,911::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >MainThread::INFO::2014-01-23 15:14:16,016::domainMonitor::125::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-01-23 15:14:16,016::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::INFO::2014-01-23 15:14:16,016::vdsm::125::vds::(run) VDSM main thread ended. Waiting for 16 other threads... >MainThread::INFO::2014-01-23 15:14:16,016::vdsm::129::vds::(run) <Thread(libvirtEventLoop, started daemon 140378863847168)> >MainThread::INFO::2014-01-23 15:14:16,016::vdsm::129::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140377717733120)> >MainThread::INFO::2014-01-23 15:14:16,016::vdsm::129::vds::(run) <WorkerThread(Thread-8, started daemon 140378321712896)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <WorkerThread(Thread-6, started daemon 140378813490944)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <HostMonitor(HostMonitor, started daemon 140377700947712)> >MainThread::INFO::2014-01-23 15:14:16,017::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <Listener(VM Channels Listener, started daemon 140377709340416)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <WorkerThread(Thread-5, started daemon 140378821883648)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <WorkerThread(Thread-4, started daemon 140378830276352)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <WorkerThread(Thread-3, started daemon 140378838669056)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <_MainThread(MainThread, started 140379075872576)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <WorkerThread(Thread-1, started daemon 140378855454464)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <GuestManager(GuestManager, started daemon 140377669273344)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <WorkerThread(Thread-10, started daemon 140378304927488)> >MainThread::INFO::2014-01-23 15:14:16,017::vdsm::129::vds::(run) <WorkerThread(Thread-2, started daemon 140378847061760)> >MainThread::INFO::2014-01-23 15:14:16,018::vdsm::129::vds::(run) <WorkerThread(Thread-9, started daemon 140378313320192)> >MainThread::INFO::2014-01-23 15:14:16,018::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:14:16,018::vdsm::129::vds::(run) <MomThread(MOM, started 140378268567296)> >MainThread::INFO::2014-01-23 15:14:16,018::vdsm::129::vds::(run) <WorkerThread(Thread-7, started daemon 140378805098240)> >VM Channels Listener::INFO::2014-01-23 15:14:16,193::vmChannels::178::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-01-23 15:17:55,441::vdsm::119::vds::(run) (PID: 2651) I am the actual vdsm 4.14.1-2.fc19 localhost.localdomain (3.12.8-200.fc19.x86_64) >MainThread::WARNING::2014-01-23 15:17:55,445::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-01-23 15:17:55,594::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-01-23 15:17:55,594::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2014-01-23 15:17:55,614::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:17:55,614::hsm::447::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-01-23 15:17:55,617::hsm::479::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-01-23 15:17:55,617::hsm::480::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-01-23 15:17:55,617::hsm::482::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-01-23 15:17:55,618::hsm::525::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-01-23 15:17:55,619::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >MainThread::INFO::2014-01-23 15:17:55,620::dispatcher::99::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-01-23 15:17:55,649::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::INFO::2014-01-23 15:17:55,682::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x24ead88>>) >MainThread::INFO::2014-01-23 15:17:55,683::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::INFO::2014-01-23 15:17:55,688::momIF::47::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-01-23 15:17:55,714::vmChannels::187::vds::(settimeout) Setting channels' timeout to 30 seconds. >storageRefresh::DEBUG::2014-01-23 15:17:55,736::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >clientIFinit::DEBUG::2014-01-23 15:17:55,743::libvirtconnection::145::root::(get) trying to connect libvirt >VM Channels Listener::INFO::2014-01-23 15:17:55,753::vmChannels::170::vds::(run) Starting VM channels listener thread. >MainThread::WARNING::2014-01-23 15:17:55,758::clientIF::181::vds::(_prepareBindings) Unable to load the json rpc server module. Please make sure it is installed. >storageRefresh::DEBUG::2014-01-23 15:17:55,799::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:17:55,803::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:17:55,838::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:17:55,839::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:17:55,862::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:17:55,863::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:17:55,911::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:17:55,911::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:17:55,933::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::WARNING::2014-01-23 15:17:55,933::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/hsm-tasks already exists >storageRefresh::DEBUG::2014-01-23 15:17:55,934::hsm::391::Storage.HSM::(storageRefresh) HSM is ready >Thread-13::DEBUG::2014-01-23 15:19:23,821::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-13::DEBUG::2014-01-23 15:19:23,862::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-14::DEBUG::2014-01-23 15:19:23,936::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-14::INFO::2014-01-23 15:19:23,936::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-14::INFO::2014-01-23 15:19:23,937::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-15::DEBUG::2014-01-23 15:20:48,515::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-15::INFO::2014-01-23 15:20:48,516::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '9a1bc201-51d5-4a33-a308-288b2a111d4b'}], options=None) >Thread-15::DEBUG::2014-01-23 15:20:48,521::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-15::DEBUG::2014-01-23 15:20:48,678::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-15::DEBUG::2014-01-23 15:20:48,685::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-15::DEBUG::2014-01-23 15:20:48,685::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-15::INFO::2014-01-23 15:20:48,685::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '9a1bc201-51d5-4a33-a308-288b2a111d4b'}]} >Thread-16::DEBUG::2014-01-23 15:20:48,695::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-16::INFO::2014-01-23 15:20:48,696::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=None, storageType=None, remotePath=None, options=None) >Thread-16::DEBUG::2014-01-23 15:20:48,696::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-16::DEBUG::2014-01-23 15:20:48,696::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-16::DEBUG::2014-01-23 15:20:48,709::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-16::DEBUG::2014-01-23 15:20:48,742::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-01-23 15:20:48,744::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-16::DEBUG::2014-01-23 15:20:48,771::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-16::INFO::2014-01-23 15:20:48,884::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-18::DEBUG::2014-01-23 15:20:48,897::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-18::INFO::2014-01-23 15:20:48,898::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '9a1bc201-51d5-4a33-a308-288b2a111d4b'}], options=None) >Thread-18::DEBUG::2014-01-23 15:20:48,898::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-18::DEBUG::2014-01-23 15:20:48,918::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-18::DEBUG::2014-01-23 15:20:48,919::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-18::DEBUG::2014-01-23 15:20:48,931::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-18::DEBUG::2014-01-23 15:20:48,960::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::INFO::2014-01-23 15:20:48,961::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '9a1bc201-51d5-4a33-a308-288b2a111d4b'}]} >MainThread::DEBUG::2014-01-23 15:21:46,617::vdsm::55::vds::(sigtermHandler) Received signal 15 >MainThread::INFO::2014-01-23 15:21:46,963::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:21:46,964::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:21:46,965::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >VM Channels Listener::INFO::2014-01-23 15:21:47,017::vmChannels::178::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-01-23 15:21:47,113::domainMonitor::125::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-01-23 15:21:47,113::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::INFO::2014-01-23 15:21:47,114::vdsm::125::vds::(run) VDSM main thread ended. Waiting for 15 other threads... >MainThread::INFO::2014-01-23 15:21:47,114::vdsm::129::vds::(run) <Thread(libvirtEventLoop, started daemon 139936989751040)> >MainThread::INFO::2014-01-23 15:21:47,114::vdsm::129::vds::(run) <PolicyEngine(PolicyEngine, started daemon 139936074299136)> >MainThread::INFO::2014-01-23 15:21:47,114::vdsm::129::vds::(run) <WorkerThread(Thread-10, started daemon 139936569206528)> >MainThread::INFO::2014-01-23 15:21:47,114::vdsm::129::vds::(run) <WorkerThread(Thread-7, started daemon 139936594384640)> >MainThread::INFO::2014-01-23 15:21:47,114::vdsm::129::vds::(run) <WorkerThread(Thread-6, started daemon 139936602777344)> >MainThread::INFO::2014-01-23 15:21:47,114::vdsm::129::vds::(run) <WorkerThread(Thread-8, started daemon 139936585991936)> >MainThread::INFO::2014-01-23 15:21:47,114::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <MomThread(MOM, started 139936057513728)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <WorkerThread(Thread-5, started daemon 139936611170048)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <_MainThread(MainThread, started 139937201776448)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <WorkerThread(Thread-4, started daemon 139936956180224)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <WorkerThread(Thread-3, started daemon 139936964572928)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <WorkerThread(Thread-2, started daemon 139936972965632)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <GuestManager(GuestManager, started daemon 139935789078272)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <WorkerThread(Thread-9, started daemon 139936577599232)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <WorkerThread(Thread-1, started daemon 139936981358336)> >MainThread::INFO::2014-01-23 15:21:47,115::vdsm::129::vds::(run) <HostMonitor(HostMonitor, started daemon 139935797470976)> >MainThread::INFO::2014-01-23 15:21:58,204::vdsm::119::vds::(run) (PID: 3144) I am the actual vdsm 4.14.1-2.fc19 localhost.localdomain (3.12.8-200.fc19.x86_64) >MainThread::WARNING::2014-01-23 15:21:58,207::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-01-23 15:21:58,347::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-01-23 15:21:58,347::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2014-01-23 15:21:58,366::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:21:58,367::hsm::447::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-01-23 15:21:58,369::hsm::479::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-01-23 15:21:58,369::hsm::480::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-01-23 15:21:58,369::hsm::482::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-01-23 15:21:58,370::hsm::525::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-01-23 15:21:58,371::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >MainThread::INFO::2014-01-23 15:21:58,372::dispatcher::99::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-01-23 15:21:58,373::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::INFO::2014-01-23 15:21:58,412::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x1f2dd40>>) >MainThread::INFO::2014-01-23 15:21:58,413::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::INFO::2014-01-23 15:21:58,414::momIF::47::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-01-23 15:21:58,422::vmChannels::187::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-01-23 15:21:58,423::vmChannels::170::vds::(run) Starting VM channels listener thread. >MainThread::WARNING::2014-01-23 15:21:58,438::clientIF::181::vds::(_prepareBindings) Unable to load the json rpc server module. Please make sure it is installed. >clientIFinit::DEBUG::2014-01-23 15:21:58,440::libvirtconnection::145::root::(get) trying to connect libvirt >Thread-13::DEBUG::2014-01-23 15:21:59,063::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-13::DEBUG::2014-01-23 15:21:59,063::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Recovering from crash or Initializing', 'code': 99}} >Thread-14::DEBUG::2014-01-23 15:22:00,074::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-14::DEBUG::2014-01-23 15:22:00,074::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Recovering from crash or Initializing', 'code': 99}} >storageRefresh::DEBUG::2014-01-23 15:22:00,397::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:22:00,442::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:22:00,444::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:22:00,463::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:22:00,464::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:22:00,483::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:22:00,484::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:22:00,500::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:22:00,520::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:22:00,535::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::WARNING::2014-01-23 15:22:00,535::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/hsm-tasks already exists >storageRefresh::DEBUG::2014-01-23 15:22:00,536::hsm::391::Storage.HSM::(storageRefresh) HSM is ready >Thread-15::DEBUG::2014-01-23 15:22:01,087::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-15::DEBUG::2014-01-23 15:22:01,106::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-16::DEBUG::2014-01-23 15:22:01,117::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getCapabilities with () {} >Thread-16::DEBUG::2014-01-23 15:22:01,173::utils::555::root::(execCmd) '/sbin/ip route show to 0.0.0.0/0 table all' (cwd None) >Thread-16::DEBUG::2014-01-23 15:22:01,186::utils::575::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-01-23 15:22:01,289::BindingXMLRPC::977::vds::(wrapper) return getCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:ae5ca5ff9aef'}], 'FC': []}, 'packages2': {'kernel': {'release': '200.fc19.x86_64', 'buildtime': 1389838691.0, 'version': '3.12.8'}, 'spice-server': {'release': '3.fc19', 'buildtime': 1383130020L, 'version': '0.12.4'}, 'vdsm': {'release': '2.fc19', 'buildtime': 1390310463L, 'version': '4.14.1'}, 'qemu-kvm': {'release': '15.fc19', 'buildtime': 1387388596L, 'version': '1.4.2'}, 'libvirt': {'release': '1.fc19', 'buildtime': 1387055699L, 'version': '1.0.5.8'}, 'qemu-img': {'release': '15.fc19', 'buildtime': 1387388596L, 'version': '1.4.2'}, 'mom': {'release': '20140120.gitfd877c5.fc19', 'buildtime': 1390225304L, 'version': '0.3.2'}}, 'cpuModel': 'Intel Core 2 Duo P9xxx (Penryn Class Core 2)', 'hooks': {'before_vm_start': {'50_hostedengine': {'md5': '45dde62155b5412eafbfff5ef265acc2'}}}, 'cpuSockets': '2', 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {}, 'bridges': {';vdsmdummy;': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'stp': 'off', 'ipv6gateway': '::', 'gateway': '', 'ports': []}}, 'uuid': '339190D0-B740-403F-9394-92C6882C4898_66:1a:4a:23:11:6c', 'lastClientIface': 'lo', 'nics': {'eth0': {'netmask': '255.255.254.0', 'addr': '10.35.7.37', 'hwaddr': '66:1a:4a:23:11:6c', 'cfg': {'PEERROUTES': 'yes', 'UUID': '5129de77-56d7-4f38-a9ba-e478d75e2a6c', 'NAME': 'eth0', 'TYPE': 'Ethernet', 'IPV6_PEERDNS': 'yes', 'DEFROUTE': 'yes', 'PEERDNS': 'yes', 'IPV4_FAILURE_FATAL': 'no', 'IPV6_AUTOCONF': 'yes', 'BOOTPROTO': 'dhcp', 'HWADDR': '66:1A:4A:23:11:9D', 'IPV6_FAILURE_FATAL': 'no', 'IPV6_PEERROUTES': 'yes', 'IPV6_DEFROUTE': 'yes', 'ONBOOT': 'yes', 'IPV6INIT': 'yes'}, 'ipv6addrs': ['fe80::641a:4aff:fe23:116c/64'], 'speed': 0, 'mtu': '1500'}}, 'software_revision': '2', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3'], 'cpuFlags': u'fpu,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pse36,clflush,mmx,fxsr,sse,sse2,syscall,nx,lm,rep_good,nopl,pni,vmx,ssse3,cx16,sse4_1,hypervisor,lahf_lm,model_Conroe,model_Penryn', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:ae5ca5ff9aef', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4'], 'reservedMem': '321', 'bondings': {'bond0': {'netmask': '', 'addr': '', 'slaves': [], 'hwaddr': 'be:0c:8e:a3:bd:89', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500'}}, 'software_version': '4.14', 'memSize': '5970', 'cpuSpeed': '3392.292', 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '2', 'kvmEnabled': 'true', 'guestOverhead': '65', 'management_ip': '0.0.0.0', 'cpuThreads': '2', 'emulatedMachines': [u'pc', u'q35', u'isapc', u'pc-0.10', u'pc-0.11', u'pc-0.12', u'pc-0.13', u'pc-0.14', u'pc-0.15', u'pc-1.0', u'pc-1.1', u'pc-1.2', u'pc-1.3', u'none'], 'rngSources': ['random'], 'operatingSystem': {'release': '4', 'version': '19', 'name': 'Fedora'}, 'lastClient': '127.0.0.1'}} >Thread-17::DEBUG::2014-01-23 15:22:01,391::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call addNetwork with ('ovirtmgmt', '', '', ['eth0'], {'blockingdhcp': 'true', 'bootproto': 'dhcp', 'force': 'False', 'ONBOOT': 'yes', 'bridged': 'True'}) {} >Thread-17::DEBUG::2014-01-23 15:22:02,198::BindingXMLRPC::977::vds::(wrapper) return addNetwork with {'status': {'message': 'Done', 'code': 0}} >Thread-18::DEBUG::2014-01-23 15:22:02,879::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-18::INFO::2014-01-23 15:22:02,879::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-18::INFO::2014-01-23 15:22:02,879::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-19::DEBUG::2014-01-23 15:22:02,889::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-19::INFO::2014-01-23 15:22:02,889::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '9a1bc201-51d5-4a33-a308-288b2a111d4b'}], options=None) >Thread-19::DEBUG::2014-01-23 15:22:02,892::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-19::ERROR::2014-01-23 15:22:02,907::hsm::2362::Storage.HSM::(connectStorageServer) Could not connect to storageServer >Traceback (most recent call last): > File "/usr/share/vdsm/storage/hsm.py", line 2359, in connectStorageServer > conObj.connect() > File "/usr/share/vdsm/storage/storageServer.py", line 320, in connect > return self._mountCon.connect() > File "/usr/share/vdsm/storage/storageServer.py", line 215, in connect > raise e >MountError: (32, ';mount.nfs: Failed to resolve server multipass.eng.lab.tlv.redhat.com: Name or service not known\n') >Thread-19::DEBUG::2014-01-23 15:22:02,932::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-19::INFO::2014-01-23 15:22:02,932::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 477, 'id': '9a1bc201-51d5-4a33-a308-288b2a111d4b'}]} >Thread-20::DEBUG::2014-01-23 15:22:02,944::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-20::INFO::2014-01-23 15:22:02,944::logUtils::44::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='8b90415e-fdf9-4380-a778-f77e0e255faa', domainName='hosted_storage', typeSpecificArg='multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', domClass=1, domVersion=3, options=None) >Thread-20::DEBUG::2014-01-23 15:22:02,945::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-20::DEBUG::2014-01-23 15:22:02,945::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-20::DEBUG::2014-01-23 15:22:02,957::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-20::DEBUG::2014-01-23 15:22:03,045::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-20::ERROR::2014-01-23 15:22:03,046::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 8b90415e-fdf9-4380-a778-f77e0e255faa >Thread-20::ERROR::2014-01-23 15:22:03,046::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 8b90415e-fdf9-4380-a778-f77e0e255faa >Thread-20::DEBUG::2014-01-23 15:22:03,047::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 8b90415e-fdf9-4380-a778-f77e0e255faa' (cwd None) >Thread-20::DEBUG::2014-01-23 15:22:03,077::lvm::295::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "8b90415e-fdf9-4380-a778-f77e0e255faa" not found\n'; <rc> = 5 >Thread-20::WARNING::2014-01-23 15:22:03,078::lvm::377::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "8b90415e-fdf9-4380-a778-f77e0e255faa" not found'] >Thread-20::ERROR::2014-01-23 15:22:03,079::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 8b90415e-fdf9-4380-a778-f77e0e255faa not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('8b90415e-fdf9-4380-a778-f77e0e255faa',) >Thread-20::INFO::2014-01-23 15:22:03,081::nfsSD::69::Storage.StorageDomain::(create) sdUUID=8b90415e-fdf9-4380-a778-f77e0e255faa domainName=hosted_storage remotePath=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday domClass=1 >Thread-20::ERROR::2014-01-23 15:22:03,084::dispatcher::67::Storage.Dispatcher.Protect::(run) {'status': {'message': "Storage domain remote path not mounted: ('/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday',)", 'code': 360}} >Thread-21::DEBUG::2014-01-23 15:23:02,425::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-21::DEBUG::2014-01-23 15:23:02,427::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-22::DEBUG::2014-01-23 15:23:02,498::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-22::INFO::2014-01-23 15:23:02,498::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-22::INFO::2014-01-23 15:23:02,498::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-23::DEBUG::2014-01-23 15:23:05,705::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-23::INFO::2014-01-23 15:23:05,706::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}], options=None) >Thread-23::DEBUG::2014-01-23 15:23:05,708::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-23::DEBUG::2014-01-23 15:23:05,898::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-23::DEBUG::2014-01-23 15:23:05,903::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-23::DEBUG::2014-01-23 15:23:05,903::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-23::INFO::2014-01-23 15:23:05,903::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}]} >Thread-24::DEBUG::2014-01-23 15:23:05,913::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-24::INFO::2014-01-23 15:23:05,914::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=None, storageType=None, remotePath=None, options=None) >Thread-24::DEBUG::2014-01-23 15:23:05,914::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-24::DEBUG::2014-01-23 15:23:05,914::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-24::DEBUG::2014-01-23 15:23:05,926::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-24::DEBUG::2014-01-23 15:23:05,972::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-24::DEBUG::2014-01-23 15:23:05,974::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-24::DEBUG::2014-01-23 15:23:06,001::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-24::INFO::2014-01-23 15:23:06,103::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': []} >Thread-26::DEBUG::2014-01-23 15:23:06,115::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-26::INFO::2014-01-23 15:23:06,116::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}], options=None) >Thread-26::DEBUG::2014-01-23 15:23:06,116::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-26::DEBUG::2014-01-23 15:23:06,142::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-26::DEBUG::2014-01-23 15:23:06,142::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-26::DEBUG::2014-01-23 15:23:06,156::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-26::DEBUG::2014-01-23 15:23:06,198::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-26::INFO::2014-01-23 15:23:06,199::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}]} >MainThread::DEBUG::2014-01-23 15:23:43,330::vdsm::55::vds::(sigtermHandler) Received signal 15 >MainThread::INFO::2014-01-23 15:23:44,152::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:23:44,153::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:23:44,154::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >MainThread::INFO::2014-01-23 15:23:44,312::domainMonitor::125::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-01-23 15:23:44,312::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::INFO::2014-01-23 15:23:44,313::vdsm::125::vds::(run) VDSM main thread ended. Waiting for 16 other threads... >MainThread::INFO::2014-01-23 15:23:44,313::vdsm::129::vds::(run) <Thread(libvirtEventLoop, started daemon 139778965960448)> >MainThread::INFO::2014-01-23 15:23:44,313::vdsm::129::vds::(run) <WorkerThread(Thread-4, started daemon 139778932389632)> >MainThread::INFO::2014-01-23 15:23:44,313::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:23:44,313::vdsm::129::vds::(run) <MomThread(MOM, started 139777898706688)> >MainThread::INFO::2014-01-23 15:23:44,313::vdsm::129::vds::(run) <GuestManager(GuestManager, started daemon 139777851213568)> >MainThread::INFO::2014-01-23 15:23:44,313::vdsm::129::vds::(run) <HostMonitor(HostMonitor, started daemon 139777867998976)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <WorkerThread(Thread-10, started daemon 139778410399488)> >MainThread::INFO::2014-01-23 15:23:44,314::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <Listener(VM Channels Listener, started daemon 139777881921280)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <WorkerThread(Thread-8, started daemon 139778427184896)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <WorkerThread(Thread-9, started daemon 139778418792192)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <_MainThread(MainThread, started 139779177985856)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <WorkerThread(Thread-3, started daemon 139778940782336)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <WorkerThread(Thread-7, started daemon 139778435577600)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <WorkerThread(Thread-6, started daemon 139778915604224)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <WorkerThread(Thread-5, started daemon 139778923996928)> >MainThread::INFO::2014-01-23 15:23:44,314::vdsm::129::vds::(run) <PolicyEngine(PolicyEngine, started daemon 139777890313984)> >MainThread::INFO::2014-01-23 15:23:44,315::vdsm::129::vds::(run) <WorkerThread(Thread-2, started daemon 139778949175040)> >MainThread::INFO::2014-01-23 15:23:44,315::vdsm::129::vds::(run) <WorkerThread(Thread-1, started daemon 139778957567744)> >VM Channels Listener::INFO::2014-01-23 15:23:44,539::vmChannels::178::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-01-23 15:23:50,439::vdsm::119::vds::(run) (PID: 4245) I am the actual vdsm 4.14.1-2.fc19 localhost.localdomain (3.12.8-200.fc19.x86_64) >MainThread::WARNING::2014-01-23 15:23:50,442::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-01-23 15:23:50,577::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-01-23 15:23:50,578::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2014-01-23 15:23:50,602::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:23:50,603::hsm::447::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-01-23 15:23:50,605::hsm::479::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-01-23 15:23:50,605::hsm::480::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-01-23 15:23:50,605::hsm::482::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-01-23 15:23:50,606::hsm::525::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-01-23 15:23:50,607::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-01-23 15:23:50,607::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::INFO::2014-01-23 15:23:50,607::dispatcher::99::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-01-23 15:23:50,622::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >MainThread::INFO::2014-01-23 15:23:50,656::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x1e57cf8>>) >MainThread::INFO::2014-01-23 15:23:50,707::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >storageRefresh::DEBUG::2014-01-23 15:23:50,669::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >MainThread::INFO::2014-01-23 15:23:50,709::momIF::47::MOM::(__init__) Starting up MOM >storageRefresh::DEBUG::2014-01-23 15:23:50,711::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >MainThread::INFO::2014-01-23 15:23:50,719::vmChannels::187::vds::(settimeout) Setting channels' timeout to 30 seconds. >clientIFinit::DEBUG::2014-01-23 15:23:50,735::libvirtconnection::145::root::(get) trying to connect libvirt >VM Channels Listener::INFO::2014-01-23 15:23:50,736::vmChannels::170::vds::(run) Starting VM channels listener thread. >MainThread::WARNING::2014-01-23 15:23:50,737::clientIF::181::vds::(_prepareBindings) Unable to load the json rpc server module. Please make sure it is installed. >storageRefresh::DEBUG::2014-01-23 15:23:50,767::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:23:50,771::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:23:50,792::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:23:50,793::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:23:50,812::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:23:50,813::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:23:50,851::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::WARNING::2014-01-23 15:23:50,902::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/hsm-tasks already exists >storageRefresh::DEBUG::2014-01-23 15:23:50,902::hsm::391::Storage.HSM::(storageRefresh) HSM is ready >Thread-13::DEBUG::2014-01-23 15:23:51,295::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-13::DEBUG::2014-01-23 15:23:51,314::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-14::DEBUG::2014-01-23 15:23:51,325::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-14::INFO::2014-01-23 15:23:51,325::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-14::INFO::2014-01-23 15:23:51,325::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-15::DEBUG::2014-01-23 15:23:51,335::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-15::INFO::2014-01-23 15:23:51,336::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}], options=None) >Thread-15::DEBUG::2014-01-23 15:23:51,339::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday' (cwd None) >Thread-15::DEBUG::2014-01-23 15:23:51,486::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-15::DEBUG::2014-01-23 15:23:51,489::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-15::DEBUG::2014-01-23 15:23:51,489::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-15::INFO::2014-01-23 15:23:51,489::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}]} >Thread-16::DEBUG::2014-01-23 15:23:51,500::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-16::INFO::2014-01-23 15:23:51,501::logUtils::44::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=1, sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', domainName='hosted_storage', typeSpecificArg='multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', domClass=1, domVersion=3, options=None) >Thread-16::DEBUG::2014-01-23 15:23:51,501::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-16::DEBUG::2014-01-23 15:23:51,501::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-16::DEBUG::2014-01-23 15:23:51,513::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-16::DEBUG::2014-01-23 15:23:51,548::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::ERROR::2014-01-23 15:23:51,549::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-16::ERROR::2014-01-23 15:23:51,549::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-16::DEBUG::2014-01-23 15:23:51,550::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name f2b9175e-84eb-4f09-81e9-8c55a60c08e5' (cwd None) >Thread-16::DEBUG::2014-01-23 15:23:51,574::lvm::295::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "f2b9175e-84eb-4f09-81e9-8c55a60c08e5" not found\n'; <rc> = 5 >Thread-16::WARNING::2014-01-23 15:23:51,629::lvm::377::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "f2b9175e-84eb-4f09-81e9-8c55a60c08e5" not found'] >Thread-16::ERROR::2014-01-23 15:23:51,739::sdc::143::Storage.StorageDomainCache::(_findDomain) domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('f2b9175e-84eb-4f09-81e9-8c55a60c08e5',) >Thread-16::INFO::2014-01-23 15:23:51,739::nfsSD::69::Storage.StorageDomain::(create) sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 domainName=hosted_storage remotePath=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday domClass=1 >Thread-16::DEBUG::2014-01-23 15:23:52,013::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-01-23 15:23:52,019::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[] >Thread-16::WARNING::2014-01-23 15:23:52,019::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is >Thread-16::DEBUG::2014-01-23 15:23:52,019::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-16::DEBUG::2014-01-23 15:23:52,019::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-16::DEBUG::2014-01-23 15:23:52,019::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Regular', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=29b05c7506fa748b674f5c214f60b635a4c0d15b'] >Thread-16::DEBUG::2014-01-23 15:23:52,046::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-16::DEBUG::2014-01-23 15:23:52,055::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-16::DEBUG::2014-01-23 15:23:52,056::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-16::DEBUG::2014-01-23 15:23:52,067::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Regular', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=29b05c7506fa748b674f5c214f60b635a4c0d15b'] >Thread-16::DEBUG::2014-01-23 15:23:52,071::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::DEBUG::2014-01-23 15:23:52,200::sd::424::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-16::DEBUG::2014-01-23 15:23:52,200::hsm::2620::Storage.HSM::(createStorageDomain) knownSDs: {f2b9175e-84eb-4f09-81e9-8c55a60c08e5: storage.nfsSD.findDomain} >Thread-16::INFO::2014-01-23 15:23:52,201::logUtils::47::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-18::DEBUG::2014-01-23 15:23:52,211::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-18::INFO::2014-01-23 15:23:52,211::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-18::INFO::2014-01-23 15:23:52,211::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-19::DEBUG::2014-01-23 15:23:52,221::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-19::INFO::2014-01-23 15:23:52,221::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', options=None) >Thread-19::INFO::2014-01-23 15:23:52,224::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '262825574400', 'disktotal': '731155267584', 'mdafree': 0}} >Thread-20::DEBUG::2014-01-23 15:23:52,235::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-20::INFO::2014-01-23 15:23:52,235::logUtils::44::dispatcher::(wrapper) Run and protect: createStoragePool(poolType=None, spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', poolName='hosted_datacenter', masterDom='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', domList=['f2b9175e-84eb-4f09-81e9-8c55a60c08e5'], masterVersion=1, lockPolicy=None, lockRenewalIntervalSec=None, leaseTimeSec=None, ioOpTimeoutSec=None, leaseRetries=None, options=None) >Thread-20::INFO::2014-01-23 15:23:52,235::fileSD::316::Storage.StorageDomain::(validate) sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-20::DEBUG::2014-01-23 15:23:52,244::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Regular', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=29b05c7506fa748b674f5c214f60b635a4c0d15b'] >Thread-20::INFO::2014-01-23 15:23:52,246::sp::133::Storage.StoragePool::(setBackend) updating pool def8e1c1-183d-4a22-82c0-483e982952f0 backend from type NoneType instance 0x3cf758bf00 to type StoragePoolDiskBackend instance 0x7fa328043cf8 >Thread-20::INFO::2014-01-23 15:23:52,246::sp::543::Storage.StoragePool::(create) spUUID=def8e1c1-183d-4a22-82c0-483e982952f0 poolName=hosted_datacenter master_sd=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 domList=['f2b9175e-84eb-4f09-81e9-8c55a60c08e5'] masterVersion=1 {'LEASETIMESEC': 60, 'LOCKPOLICY': 'ON', 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3, 'LOCKRENEWALINTERVALSEC': 5} >Thread-20::INFO::2014-01-23 15:23:52,246::fileSD::316::Storage.StorageDomain::(validate) sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-20::DEBUG::2014-01-23 15:23:52,254::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Regular', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=29b05c7506fa748b674f5c214f60b635a4c0d15b'] >Thread-20::DEBUG::2014-01-23 15:23:52,262::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Regular', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=29b05c7506fa748b674f5c214f60b635a4c0d15b'] >Thread-20::DEBUG::2014-01-23 15:23:52,262::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-20::DEBUG::2014-01-23 15:23:52,262::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-20::DEBUG::2014-01-23 15:23:52,262::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Regular', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=ace49021d4baa7a4dbfaebeb50244cf56e8d2e6d'] >Thread-20::DEBUG::2014-01-23 15:23:52,301::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-20::INFO::2014-01-23 15:24:13,452::sp::666::Storage.StoragePool::(createMaster) setting master domain for spUUID def8e1c1-183d-4a22-82c0-483e982952f0 on sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-20::DEBUG::2014-01-23 15:24:13,452::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-20::DEBUG::2014-01-23 15:24:13,452::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-20::DEBUG::2014-01-23 15:24:13,452::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Regular', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=c1ef1160174d662e97242d4e5a038bd754cb21ae'] >Thread-20::DEBUG::2014-01-23 15:24:13,493::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-20::DEBUG::2014-01-23 15:24:13,502::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Regular', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=c1ef1160174d662e97242d4e5a038bd754cb21ae'] >Thread-20::DEBUG::2014-01-23 15:24:13,503::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-20::DEBUG::2014-01-23 15:24:13,503::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-20::DEBUG::2014-01-23 15:24:13,503::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-20::DEBUG::2014-01-23 15:24:13,503::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-20::DEBUG::2014-01-23 15:24:13,503::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-20::DEBUG::2014-01-23 15:24:13,503::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >Thread-20::DEBUG::2014-01-23 15:24:13,538::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-20::DEBUG::2014-01-23 15:24:13,538::sp::1366::Storage.StoragePool::(setMasterDomain) Master domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 verified, version 1 >Thread-20::DEBUG::2014-01-23 15:24:13,547::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >Thread-20::INFO::2014-01-23 15:24:13,548::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-21::DEBUG::2014-01-23 15:24:13,548::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-20::DEBUG::2014-01-23 15:24:13,548::sp::1395::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `def8e1c1-183d-4a22-82c0-483e982952f0` started monitoring domain `f2b9175e-84eb-4f09-81e9-8c55a60c08e5` >Thread-20::WARNING::2014-01-23 15:24:13,549::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0 already exists >Thread-20::DEBUG::2014-01-23 15:24:13,560::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >Thread-20::DEBUG::2014-01-23 15:24:13,561::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-20::DEBUG::2014-01-23 15:24:13,618::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-20::INFO::2014-01-23 15:24:13,625::sp::1104::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 to /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-20::INFO::2014-01-23 15:24:13,625::sp::1104::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 to /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/mastersd >Thread-21::DEBUG::2014-01-23 15:24:13,664::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-21::DEBUG::2014-01-23 15:24:13,711::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n521 bytes (521 B) copied, 0.00628745 s, 82.9 kB/s\n'; <rc> = 0 >Thread-21::DEBUG::2014-01-23 15:24:14,160::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 changed its status to Valid >Thread-20::DEBUG::2014-01-23 15:24:17,172::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >Thread-20::INFO::2014-01-23 15:24:17,172::domainMonitor::113::Storage.DomainMonitor::(stopMonitoring) Stop monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-21::DEBUG::2014-01-23 15:24:17,173::domainMonitor::175::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-20::INFO::2014-01-23 15:24:17,173::logUtils::47::dispatcher::(wrapper) Run and protect: createStoragePool, Return response: True >Thread-23::DEBUG::2014-01-23 15:24:17,184::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-23::INFO::2014-01-23 15:24:17,185::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', hostID=1, msdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', masterVersion=1, domainsMap=None, options=None) >Thread-23::INFO::2014-01-23 15:24:17,186::sp::133::Storage.StoragePool::(setBackend) updating pool def8e1c1-183d-4a22-82c0-483e982952f0 backend from type NoneType instance 0x3cf758bf00 to type StoragePoolDiskBackend instance 0x7fa32016be18 >Thread-23::INFO::2014-01-23 15:24:17,186::sp::618::Storage.StoragePool::(connect) Connect host #1 to the storage pool def8e1c1-183d-4a22-82c0-483e982952f0 with master domain: f2b9175e-84eb-4f09-81e9-8c55a60c08e5 (ver = 1) >Thread-23::DEBUG::2014-01-23 15:24:17,187::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-23::DEBUG::2014-01-23 15:24:17,187::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-23::DEBUG::2014-01-23 15:24:17,198::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-23::DEBUG::2014-01-23 15:24:17,234::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-23::DEBUG::2014-01-23 15:24:17,240::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-23::DEBUG::2014-01-23 15:24:17,242::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-23::DEBUG::2014-01-23 15:24:17,250::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >Thread-23::DEBUG::2014-01-23 15:24:17,251::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-23::INFO::2014-01-23 15:24:17,251::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_imageNS already registered >Thread-23::INFO::2014-01-23 15:24:17,251::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_volumeNS already registered >Thread-23::DEBUG::2014-01-23 15:24:17,252::sp::1366::Storage.StoragePool::(setMasterDomain) Master domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 verified, version 1 >Thread-23::DEBUG::2014-01-23 15:24:17,259::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >Thread-23::INFO::2014-01-23 15:24:17,259::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-25::DEBUG::2014-01-23 15:24:17,260::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-23::DEBUG::2014-01-23 15:24:17,260::sp::1395::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `def8e1c1-183d-4a22-82c0-483e982952f0` started monitoring domain `f2b9175e-84eb-4f09-81e9-8c55a60c08e5` >Thread-23::WARNING::2014-01-23 15:24:17,261::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0 already exists >Thread-23::DEBUG::2014-01-23 15:24:17,274::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >Thread-23::DEBUG::2014-01-23 15:24:17,281::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-25::DEBUG::2014-01-23 15:24:17,289::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-23::DEBUG::2014-01-23 15:24:17,299::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-25::DEBUG::2014-01-23 15:24:17,302::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n521 bytes (521 B) copied, 0.000743102 s, 701 kB/s\n'; <rc> = 0 >Thread-23::INFO::2014-01-23 15:24:17,304::sp::1104::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 to /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-23::INFO::2014-01-23 15:24:17,347::sp::1104::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 to /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/mastersd >Thread-23::INFO::2014-01-23 15:24:17,347::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-25::DEBUG::2014-01-23 15:24:17,352::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 changed its status to Valid >Thread-27::DEBUG::2014-01-23 15:24:17,352::misc::884::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-27::DEBUG::2014-01-23 15:24:17,352::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-27::DEBUG::2014-01-23 15:24:17,353::misc::904::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-29::DEBUG::2014-01-23 15:24:17,358::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-29::INFO::2014-01-23 15:24:17,359::logUtils::44::dispatcher::(wrapper) Run and protect: spmStart(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', prevID=-1, prevLVER=-1, maxHostID=250, domVersion=3, options=None) >Thread-29::INFO::2014-01-23 15:24:17,359::logUtils::47::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:17,368::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >Thread-30::DEBUG::2014-01-23 15:24:17,372::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-30::INFO::2014-01-23 15:24:17,373::logUtils::44::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-28::INFO::2014-01-23 15:24:17,394::clientIF::126::vds::(contEIOVms) vmContainerLock acquired >Thread-25::DEBUG::2014-01-23 15:24:27,362::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-25::DEBUG::2014-01-23 15:24:27,370::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n521 bytes (521 B) copied, 0.00049935 s, 1.0 MB/s\n'; <rc> = 0 >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,569::sp::266::Storage.StoragePool::(startSpm) spm lock acquired successfully >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,578::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=65a7b8f05c71c1b8d961dfdc9e2c1a4ca51de5d3'] >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,578::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,579::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,579::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f3104774924c5313d393ba743b06d434808b2083'] >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,612::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >6c7d9872-644d-448b-8f86-06a45d529f5d::INFO::2014-01-23 15:24:38,613::sp::431::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `f2b9175e-84eb-4f09-81e9-8c55a60c08e5` >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,620::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,622::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,630::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f3104774924c5313d393ba743b06d434808b2083'] >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,632::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >6c7d9872-644d-448b-8f86-06a45d529f5d::INFO::2014-01-23 15:24:38,632::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_imageNS already registered >6c7d9872-644d-448b-8f86-06a45d529f5d::INFO::2014-01-23 15:24:38,633::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_volumeNS already registered >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,633::sp::436::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,639::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f3104774924c5313d393ba743b06d434808b2083'] >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,640::sp::443::Storage.StoragePool::(_upgradePool) Registering with state change event >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,640::sp::446::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,651::sd::623::Storage.StorageDomain::(createMasterTree) creating vms dir: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/master/vms >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,659::sd::626::Storage.StorageDomain::(createMasterTree) creating task dir: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/master/tasks >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,678::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f3104774924c5313d393ba743b06d434808b2083'] >6c7d9872-644d-448b-8f86-06a45d529f5d::DEBUG::2014-01-23 15:24:38,680::sp::326::Storage.StoragePool::(startSpm) ended. >Thread-30::INFO::2014-01-23 15:24:38,682::sp::1015::Storage.StoragePool::(activateSD) sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 spUUID=def8e1c1-183d-4a22-82c0-483e982952f0 >Thread-30::INFO::2014-01-23 15:24:38,682::logUtils::47::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-32::DEBUG::2014-01-23 15:24:38,694::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-32::INFO::2014-01-23 15:24:38,694::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-32::INFO::2014-01-23 15:24:38,694::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-33::DEBUG::2014-01-23 15:24:38,703::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-33::INFO::2014-01-23 15:24:38,704::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-33::INFO::2014-01-23 15:24:38,727::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-34::DEBUG::2014-01-23 15:24:38,737::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-34::INFO::2014-01-23 15:24:38,737::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-34::INFO::2014-01-23 15:24:38,741::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'hosted_datacenter', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'master_uuid': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'status': 'Active', 'diskfree': '262823477248', 'isoprefix': '', 'alerts': [], 'disktotal': '731154219008', 'version': 3}}} >Thread-35::DEBUG::2014-01-23 15:24:38,752::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-35::INFO::2014-01-23 15:24:38,752::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-35::INFO::2014-01-23 15:24:38,752::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00049935', 'lastCheck': '0.1', 'valid': True}} >Thread-36::DEBUG::2014-01-23 15:24:38,994::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-36::INFO::2014-01-23 15:24:38,995::logUtils::44::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', imgUUID='d3ba7005-ddf4-4016-ab02-32790a5203d3', size=52428800, volFormat=5, preallocate=2, diskType=2, volUUID='5e499db1-444a-43a3-ba15-a16c382d7359', desc='Hosted Engine Image', srcImgUUID='00000000-0000-0000-0000-000000000000', srcVolUUID='00000000-0000-0000-0000-000000000000') >Thread-36::INFO::2014-01-23 15:24:39,183::logUtils::47::dispatcher::(wrapper) Run and protect: createVolume, Return response: None >Thread-37::DEBUG::2014-01-23 15:24:39,480::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-37::INFO::2014-01-23 15:24:39,481::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-37::INFO::2014-01-23 15:24:39,481::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'f6aeda9b-d28b-4bca-8608-50974cbfd006': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'f6aeda9b-d28b-4bca-8608-50974cbfd006'}}} >f6aeda9b-d28b-4bca-8608-50974cbfd006::DEBUG::2014-01-23 15:24:39,584::resourceFactories::125::Storage.ResourcesFactories::(__getResourceCandidatesList) Image d3ba7005-ddf4-4016-ab02-32790a5203d3 does not exist in domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >f6aeda9b-d28b-4bca-8608-50974cbfd006::INFO::2014-01-23 15:24:39,585::image::123::Storage.Image::(create) Create placeholder /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3 for image's volumes >f6aeda9b-d28b-4bca-8608-50974cbfd006::INFO::2014-01-23 15:24:39,719::volume::443::Storage.Volume::(create) Creating volume 5e499db1-444a-43a3-ba15-a16c382d7359 >f6aeda9b-d28b-4bca-8608-50974cbfd006::INFO::2014-01-23 15:24:40,104::fileVolume::144::Storage.Volume::(_create) Request to create RAW volume /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359 with size = 52428800 sectors >f6aeda9b-d28b-4bca-8608-50974cbfd006::DEBUG::2014-01-23 15:24:40,372::fileVolume::395::Storage.Volume::(newVolumeLease) Initializing volume lease volUUID=5e499db1-444a-43a3-ba15-a16c382d7359 sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5, metaId=('/rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359',) >Thread-38::DEBUG::2014-01-23 15:24:40,493::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-38::INFO::2014-01-23 15:24:40,493::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-38::INFO::2014-01-23 15:24:40,494::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'f6aeda9b-d28b-4bca-8608-50974cbfd006': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'f6aeda9b-d28b-4bca-8608-50974cbfd006'}}} >Thread-39::DEBUG::2014-01-23 15:24:41,505::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-39::INFO::2014-01-23 15:24:41,506::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-39::INFO::2014-01-23 15:24:41,506::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'f6aeda9b-d28b-4bca-8608-50974cbfd006': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'f6aeda9b-d28b-4bca-8608-50974cbfd006'}}} >Thread-40::DEBUG::2014-01-23 15:24:41,515::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-40::INFO::2014-01-23 15:24:41,515::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID='f6aeda9b-d28b-4bca-8608-50974cbfd006', spUUID=None, options=None) >Thread-40::INFO::2014-01-23 15:24:41,573::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-41::DEBUG::2014-01-23 15:24:41,584::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-41::INFO::2014-01-23 15:24:41,584::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-41::INFO::2014-01-23 15:24:41,584::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-42::DEBUG::2014-01-23 15:24:41,593::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-42::INFO::2014-01-23 15:24:41,594::logUtils::44::dispatcher::(wrapper) Run and protect: spmStop(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-42::DEBUG::2014-01-23 15:24:41,594::sp::336::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process >Thread-42::DEBUG::2014-01-23 15:24:41,595::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-42::DEBUG::2014-01-23 15:24:41,596::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-42::DEBUG::2014-01-23 15:24:41,596::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-42::DEBUG::2014-01-23 15:24:41,615::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-42::INFO::2014-01-23 15:24:41,626::logUtils::47::dispatcher::(wrapper) Run and protect: spmStop, Return response: None >Thread-43::DEBUG::2014-01-23 15:24:41,636::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-43::INFO::2014-01-23 15:24:41,636::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', hostID=1, remove=False, options=None) >Thread-43::INFO::2014-01-23 15:24:41,638::sp::644::Storage.StoragePool::(disconnect) Disconnect from the storage pool def8e1c1-183d-4a22-82c0-483e982952f0 >Thread-43::DEBUG::2014-01-23 15:24:41,647::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-43::INFO::2014-01-23 15:24:41,647::domainMonitor::113::Storage.DomainMonitor::(stopMonitoring) Stop monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-25::DEBUG::2014-01-23 15:24:41,647::domainMonitor::175::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-43::INFO::2014-01-23 15:24:43,648::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: True >Thread-44::DEBUG::2014-01-23 15:24:43,658::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-44::INFO::2014-01-23 15:24:43,659::logUtils::44::dispatcher::(wrapper) Run and protect: startMonitoringDomain(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', hostID=1, options=None) >Thread-44::INFO::2014-01-23 15:24:43,659::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-45::DEBUG::2014-01-23 15:24:43,660::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-44::INFO::2014-01-23 15:24:43,660::logUtils::47::dispatcher::(wrapper) Run and protect: startMonitoringDomain, Return response: None >Thread-45::DEBUG::2014-01-23 15:24:43,673::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-45::DEBUG::2014-01-23 15:24:43,679::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.000508061 s, 1.0 MB/s\n'; <rc> = 0 >Thread-45::DEBUG::2014-01-23 15:24:43,682::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 changed its status to Valid >Thread-46::DEBUG::2014-01-23 15:24:43,682::misc::884::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-46::DEBUG::2014-01-23 15:24:43,682::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-46::DEBUG::2014-01-23 15:24:43,684::misc::904::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-47::INFO::2014-01-23 15:24:43,685::clientIF::126::vds::(contEIOVms) vmContainerLock acquired >Thread-48::INFO::2014-01-23 15:24:48,673::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-48::INFO::2014-01-23 15:24:48,713::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000508061', 'lastCheck': '5.0', 'valid': True}} >Thread-48::ERROR::2014-01-23 15:24:48,714::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 198, in get_local_host_score > self._config = config.Config() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 57, in __init__ > self._load(Config.static_files) > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in _load > with open(fname, 'r') as f: >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-45::DEBUG::2014-01-23 15:24:53,694::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-45::DEBUG::2014-01-23 15:24:53,728::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.00109141 s, 476 kB/s\n'; <rc> = 0 >Thread-49::INFO::2014-01-23 15:24:53,806::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49::INFO::2014-01-23 15:24:53,806::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00109141', 'lastCheck': '0.1', 'valid': True}} >Thread-49::ERROR::2014-01-23 15:24:53,807::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 198, in get_local_host_score > self._config = config.Config() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 57, in __init__ > self._load(Config.static_files) > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in _load > with open(fname, 'r') as f: >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-50::INFO::2014-01-23 15:24:58,822::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-50::INFO::2014-01-23 15:24:58,863::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00109141', 'lastCheck': '5.1', 'valid': True}} >Thread-50::ERROR::2014-01-23 15:24:58,864::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 198, in get_local_host_score > self._config = config.Config() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 57, in __init__ > self._load(Config.static_files) > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in _load > with open(fname, 'r') as f: >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-45::DEBUG::2014-01-23 15:25:03,742::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-45::DEBUG::2014-01-23 15:25:03,748::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.00111023 s, 468 kB/s\n'; <rc> = 0 >Thread-51::INFO::2014-01-23 15:25:03,884::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-51::INFO::2014-01-23 15:25:03,885::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00111023', 'lastCheck': '0.1', 'valid': True}} >Thread-51::ERROR::2014-01-23 15:25:03,885::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 198, in get_local_host_score > self._config = config.Config() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 57, in __init__ > self._load(Config.static_files) > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in _load > with open(fname, 'r') as f: >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-52::INFO::2014-01-23 15:25:08,906::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-52::INFO::2014-01-23 15:25:08,906::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00111023', 'lastCheck': '5.2', 'valid': True}} >Thread-52::ERROR::2014-01-23 15:25:08,907::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 198, in get_local_host_score > self._config = config.Config() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 57, in __init__ > self._load(Config.static_files) > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in _load > with open(fname, 'r') as f: >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-45::DEBUG::2014-01-23 15:25:13,760::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-45::DEBUG::2014-01-23 15:25:13,769::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.00109085 s, 477 kB/s\n'; <rc> = 0 >Thread-53::INFO::2014-01-23 15:25:13,926::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-53::INFO::2014-01-23 15:25:13,926::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00109085', 'lastCheck': '0.2', 'valid': True}} >Thread-53::ERROR::2014-01-23 15:25:13,927::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 198, in get_local_host_score > self._config = config.Config() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 57, in __init__ > self._load(Config.static_files) > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 63, in _load > with open(fname, 'r') as f: >IOError: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/hosted-engine.conf' >Thread-54::DEBUG::2014-01-23 15:25:20,161::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmGetStats with ('8086ca4f-c231-4955-8149-88133673bc7f',) {} >Thread-54::DEBUG::2014-01-23 15:25:20,162::BindingXMLRPC::977::vds::(wrapper) return vmGetStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}} >Thread-55::DEBUG::2014-01-23 15:25:20,294::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmCreate with ({'emulatedMachine': 'pc', 'vmId': '8086ca4f-c231-4955-8149-88133673bc7f', 'devices': [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': '84bee1ea-913a-4fb8-84ca-11c4496e5737', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'format': 'raw', 'optional': 'false', 'poolID': '00000000-0000-0000-0000-000000000000', 'volumeID': '5e499db1-444a-43a3-ba15-a16c382d7359', 'imageID': 'd3ba7005-ddf4-4016-ab02-32790a5203d3', 'specParams': {}, 'readonly': 'false', 'domainID': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'deviceId': 'd3ba7005-ddf4-4016-ab02-32790a5203d3', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:74:b6:e0', 'linkActive': 'true', 'network': 'ovirtmgmt', 'bootOrder': '1', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': '74abceb2-5e28-4449-832f-64c297e3364c', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '61b83184-6d26-4c52-8fad-fa50168c579c', 'alias': 'console0'}], 'smp': '2', 'memSize': '4096', 'cpuType': 'Penryn', 'spiceSecureChannels': 'smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir', 'vmName': 'HostedEngine', 'display': 'vnc'},) {} >Thread-55::INFO::2014-01-23 15:25:20,300::API::668::vds::(_getNetworkIp) network None: using 0 >Thread-55::INFO::2014-01-23 15:25:20,300::clientIF::361::vds::(createVm) vmContainerLock acquired by vm 8086ca4f-c231-4955-8149-88133673bc7f >Thread-55::DEBUG::2014-01-23 15:25:20,300::libvirtconnection::145::root::(get) trying to connect libvirt >Thread-56::DEBUG::2014-01-23 15:25:20,319::vm::2226::vm.Vm::(_startUnderlyingVm) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::Start >Thread-55::DEBUG::2014-01-23 15:25:20,319::clientIF::374::vds::(createVm) Total desktops after creation of 8086ca4f-c231-4955-8149-88133673bc7f is 1 >Thread-55::DEBUG::2014-01-23 15:25:20,319::BindingXMLRPC::977::vds::(wrapper) return vmCreate with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'nicModel': 'rtl8139,pv', 'vmId': '8086ca4f-c231-4955-8149-88133673bc7f', 'emulatedMachine': 'pc', 'displayIp': '0', 'pid': '0', 'devices': [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': '84bee1ea-913a-4fb8-84ca-11c4496e5737', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'format': 'raw', 'optional': 'false', 'poolID': '00000000-0000-0000-0000-000000000000', 'volumeID': '5e499db1-444a-43a3-ba15-a16c382d7359', 'imageID': 'd3ba7005-ddf4-4016-ab02-32790a5203d3', 'specParams': {}, 'readonly': 'false', 'domainID': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'deviceId': 'd3ba7005-ddf4-4016-ab02-32790a5203d3', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:74:b6:e0', 'linkActive': 'true', 'network': 'ovirtmgmt', 'bootOrder': '1', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': '74abceb2-5e28-4449-832f-64c297e3364c', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '61b83184-6d26-4c52-8fad-fa50168c579c', 'alias': 'console0'}], 'smp': '2', 'vmType': 'kvm', 'displaySecurePort': '-1', 'memSize': '4096', 'displayPort': '-1', 'cpuType': 'Penryn', 'spiceSecureChannels': 'smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir', 'clientIp': '', 'vmName': 'HostedEngine', 'display': 'vnc'}} >Thread-56::DEBUG::2014-01-23 15:25:20,319::vm::2230::vm.Vm::(_startUnderlyingVm) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::_ongoingCreations acquired >Thread-56::INFO::2014-01-23 15:25:20,321::vm::3091::vm.Vm::(_run) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::VM wrapper has started >Thread-56::INFO::2014-01-23 15:25:20,328::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', spUUID='00000000-0000-0000-0000-000000000000', imgUUID='d3ba7005-ddf4-4016-ab02-32790a5203d3', volUUID='5e499db1-444a-43a3-ba15-a16c382d7359', options=None) >Thread-56::INFO::2014-01-23 15:25:20,333::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '512', 'apparentsize': '26843545600'} >Thread-56::INFO::2014-01-23 15:25:20,333::clientIF::320::vds::(prepareVolumePath) prepared volume path: >Thread-56::INFO::2014-01-23 15:25:20,333::logUtils::44::dispatcher::(wrapper) Run and protect: prepareImage(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', spUUID='00000000-0000-0000-0000-000000000000', imgUUID='d3ba7005-ddf4-4016-ab02-32790a5203d3', leafUUID='5e499db1-444a-43a3-ba15-a16c382d7359') >Thread-56::DEBUG::2014-01-23 15:25:20,340::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for 5e499db1-444a-43a3-ba15-a16c382d7359 >Thread-56::INFO::2014-01-23 15:25:20,342::logUtils::47::dispatcher::(wrapper) Run and protect: prepareImage, Return response: {'info': {'domainID': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'volType': 'path', 'leaseOffset': 0, 'path': '/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359', 'volumeID': '5e499db1-444a-43a3-ba15-a16c382d7359', 'leasePath': '/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359.lease', 'imageID': 'd3ba7005-ddf4-4016-ab02-32790a5203d3'}, 'path': '/var/run/vdsm/storage/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359', 'imgVolumesInfo': [{'domainID': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'volType': 'path', 'leaseOffset': 0, 'path': '/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359', 'volumeID': '5e499db1-444a-43a3-ba15-a16c382d7359', 'leasePath': '/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359.lease', 'imageID': 'd3ba7005-ddf4-4016-ab02-32790a5203d3'}]} >Thread-56::INFO::2014-01-23 15:25:20,343::clientIF::320::vds::(prepareVolumePath) prepared volume path: /var/run/vdsm/storage/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359 >Thread-56::DEBUG::2014-01-23 15:25:20,368::utils::555::root::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_hostedengine' (cwd None) >Thread-58::DEBUG::2014-01-23 15:25:20,427::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-58::ERROR::2014-01-23 15:25:20,434::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-56::DEBUG::2014-01-23 15:25:20,547::utils::575::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-56::INFO::2014-01-23 15:25:20,548::hooks::100::root::(_runHooksDir) >Thread-56::DEBUG::2014-01-23 15:25:20,548::vm::3130::vm.Vm::(_run) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::<?xml version="1.0" encoding="utf-8"?><domain type="kvm"> > <name>HostedEngine</name> > <uuid>8086ca4f-c231-4955-8149-88133673bc7f</uuid> > <memory>4194304</memory> > <currentMemory>4194304</currentMemory> > <vcpu current="2">2</vcpu> > <memtune> > <min_guarantee>0</min_guarantee> > </memtune> > <devices> > <channel type="unix"> > <target name="com.redhat.rhevm.vdsm" type="virtio"/> > <source mode="bind" path="/var/lib/libvirt/qemu/channels/8086ca4f-c231-4955-8149-88133673bc7f.com.redhat.rhevm.vdsm"/> > </channel> > <channel type="unix"> > <target name="org.qemu.guest_agent.0" type="virtio"/> > <source mode="bind" path="/var/lib/libvirt/qemu/channels/8086ca4f-c231-4955-8149-88133673bc7f.org.qemu.guest_agent.0"/> > </channel> > <input bus="ps2" type="mouse"/> > <graphics autoport="yes" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" type="vnc"/> > <console type="pty"> > <target port="0" type="virtio"/> > </console> > <memballoon model="none"/> > <controller model="virtio-scsi" type="scsi"/> > <interface type="bridge"> > <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci"/> > <mac address="00:16:3e:74:b6:e0"/> > <model type="virtio"/> > <source bridge="ovirtmgmt"/> > <filterref filter="vdsm-no-mac-spoofing"/> > <link state="up"/> > <boot order="1"/> > </interface> > <disk device="cdrom" snapshot="no" type="file"> > <address bus="1" controller="0" target="0" type="drive" unit="0"/> > <source file="" startupPolicy="optional"/> > <target bus="ide" dev="hdc"/> > <readonly/> > <serial/> > </disk> > <disk device="disk" snapshot="no" type="file"> > <address bus="0x00" domain="0x0000" function="0x0" slot="0x06" type="pci"/> > <source file="/var/run/vdsm/storage/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359"/> > <target bus="virtio" dev="vda"/> > <serial>d3ba7005-ddf4-4016-ab02-32790a5203d3</serial> > <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> > </disk> > <lease> > <key>5e499db1-444a-43a3-ba15-a16c382d7359</key> > <lockspace>f2b9175e-84eb-4f09-81e9-8c55a60c08e5</lockspace> > <target offset="0" path="/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359.lease"/> > </lease> > </devices> > <os> > <type arch="x86_64" machine="pc">hvm</type> > <smbios mode="sysinfo"/> > </os> > <sysinfo type="smbios"> > <system> > <entry name="manufacturer">oVirt</entry> > <entry name="product">oVirt Node</entry> > <entry name="version">19-4</entry> > <entry name="serial">339190D0-B740-403F-9394-92C6882C4898_66:1a:4a:23:11:6c</entry> > <entry name="uuid">8086ca4f-c231-4955-8149-88133673bc7f</entry> > </system> > </sysinfo> > <clock adjustment="0" offset="variable"> > <timer name="rtc" tickpolicy="catchup"/> > </clock> > <features> > <acpi/> > </features> > <cpu match="exact"> > <model>Penryn</model> > </cpu> ><on_poweroff>destroy</on_poweroff><on_reboot>destroy</on_reboot><on_crash>destroy</on_crash></domain> >Thread-59::DEBUG::2014-01-23 15:25:21,622::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-59::ERROR::2014-01-23 15:25:21,622::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-60::DEBUG::2014-01-23 15:25:22,728::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-60::ERROR::2014-01-23 15:25:22,728::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-45::DEBUG::2014-01-23 15:25:23,785::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-45::DEBUG::2014-01-23 15:25:23,826::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.00104647 s, 497 kB/s\n'; <rc> = 0 >Thread-61::DEBUG::2014-01-23 15:25:23,834::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-61::ERROR::2014-01-23 15:25:23,834::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-62::DEBUG::2014-01-23 15:25:24,946::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-62::ERROR::2014-01-23 15:25:24,946::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-56::DEBUG::2014-01-23 15:25:25,910::libvirtconnection::124::root::(wrapper) Unknown libvirterror: ecode: 63 edom: 33 level: 2 message: Error while building firewall: Some rules could not be created for interface vnet0: Failure to execute command '$EBT -t nat -N libvirt-J-vnet0' : '[91mFirewallD is not running[00m'. > >Thread-56::DEBUG::2014-01-23 15:25:25,911::vm::2247::vm.Vm::(_startUnderlyingVm) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::_ongoingCreations released >Thread-56::ERROR::2014-01-23 15:25:25,911::vm::2273::vm.Vm::(_startUnderlyingVm) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::The vm start process failed >Traceback (most recent call last): > File "/usr/share/vdsm/vm.py", line 2233, in _startUnderlyingVm > self._run() > File "/usr/share/vdsm/vm.py", line 3164, in _run > self._connection.createXML(domxml, flags), > File "/usr/lib64/python2.7/site-packages/vdsm/libvirtconnection.py", line 92, in wrapper > ret = f(*args, **kwargs) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2805, in createXML > if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) >libvirtError: Error while building firewall: Some rules could not be created for interface vnet0: Failure to execute command '$EBT -t nat -N libvirt-J-vnet0' : '[91mFirewallD is not running[00m'. > >Thread-56::DEBUG::2014-01-23 15:25:25,923::vm::2725::vm.Vm::(setDownStatus) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::Changed state to Down: Error while building firewall: Some rules could not be created for interface vnet0: Failure to execute command '$EBT -t nat -N libvirt-J-vnet0' : '[91mFirewallD is not running[00m'. > >Thread-63::DEBUG::2014-01-23 15:25:26,050::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-63::ERROR::2014-01-23 15:25:26,050::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-64::DEBUG::2014-01-23 15:25:27,151::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-64::ERROR::2014-01-23 15:25:27,151::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-65::DEBUG::2014-01-23 15:25:28,251::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-65::ERROR::2014-01-23 15:25:28,251::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-66::DEBUG::2014-01-23 15:25:29,355::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-66::ERROR::2014-01-23 15:25:29,355::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-67::DEBUG::2014-01-23 15:25:30,455::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '8738QVnC', '10800', 'disconnect', {}) {} >Thread-67::ERROR::2014-01-23 15:25:30,455::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-68::DEBUG::2014-01-23 15:25:31,525::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-68::INFO::2014-01-23 15:25:31,525::logUtils::44::dispatcher::(wrapper) Run and protect: stopMonitoringDomain(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', options=None) >Thread-68::INFO::2014-01-23 15:25:31,526::domainMonitor::113::Storage.DomainMonitor::(stopMonitoring) Stop monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-45::DEBUG::2014-01-23 15:25:31,526::domainMonitor::175::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-68::INFO::2014-01-23 15:25:33,527::logUtils::47::dispatcher::(wrapper) Run and protect: stopMonitoringDomain, Return response: None >Thread-69::DEBUG::2014-01-23 15:32:24,946::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-69::DEBUG::2014-01-23 15:32:24,949::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-70::DEBUG::2014-01-23 15:32:25,019::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-70::INFO::2014-01-23 15:32:25,020::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-70::INFO::2014-01-23 15:32:25,020::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-71::DEBUG::2014-01-23 15:32:25,385::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-71::INFO::2014-01-23 15:32:25,386::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}], options=None) >Thread-71::DEBUG::2014-01-23 15:32:25,389::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-71::DEBUG::2014-01-23 15:32:25,396::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: ('f2b9175e-84eb-4f09-81e9-8c55a60c08e5',) >Thread-71::DEBUG::2014-01-23 15:32:25,397::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {f2b9175e-84eb-4f09-81e9-8c55a60c08e5: storage.nfsSD.findDomain} >Thread-71::INFO::2014-01-23 15:32:25,397::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}]} >Thread-72::DEBUG::2014-01-23 15:32:25,407::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-72::INFO::2014-01-23 15:32:25,407::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=None, storageType=None, remotePath=None, options=None) >Thread-72::DEBUG::2014-01-23 15:32:25,407::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-72::DEBUG::2014-01-23 15:32:25,407::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-72::DEBUG::2014-01-23 15:32:25,422::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-72::DEBUG::2014-01-23 15:32:25,480::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-01-23 15:32:25,482::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-72::DEBUG::2014-01-23 15:32:25,505::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-72::DEBUG::2014-01-23 15:32:25,513::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-72::DEBUG::2014-01-23 15:32:25,515::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-72::DEBUG::2014-01-23 15:32:25,525::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-72::DEBUG::2014-01-23 15:32:25,527::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-72::INFO::2014-01-23 15:32:25,527::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_imageNS already registered >Thread-72::INFO::2014-01-23 15:32:25,527::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_volumeNS already registered >Thread-72::INFO::2014-01-23 15:32:25,527::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainsList, Return response: {'domlist': ['f2b9175e-84eb-4f09-81e9-8c55a60c08e5']} >Thread-75::DEBUG::2014-01-23 15:32:25,538::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-75::INFO::2014-01-23 15:32:25,539::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainInfo(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', options=None) >Thread-75::INFO::2014-01-23 15:32:25,539::fileSD::316::Storage.StorageDomain::(validate) sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-75::DEBUG::2014-01-23 15:32:25,547::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-75::INFO::2014-01-23 15:32:25,551::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainInfo, Return response: {'info': {'uuid': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'version': '3', 'role': 'Master', 'remotePath': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'type': 'NFS', 'class': 'Data', 'pool': ['def8e1c1-183d-4a22-82c0-483e982952f0'], 'name': 'hosted_storage'}} >Thread-76::DEBUG::2014-01-23 15:32:32,737::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-76::INFO::2014-01-23 15:32:32,737::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', hostID=1, msdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', masterVersion=1, domainsMap=None, options=None) >Thread-76::INFO::2014-01-23 15:32:32,739::sp::133::Storage.StoragePool::(setBackend) updating pool def8e1c1-183d-4a22-82c0-483e982952f0 backend from type NoneType instance 0x3cf758bf00 to type StoragePoolDiskBackend instance 0x7fa3280450e0 >Thread-76::INFO::2014-01-23 15:32:32,739::sp::618::Storage.StoragePool::(connect) Connect host #1 to the storage pool def8e1c1-183d-4a22-82c0-483e982952f0 with master domain: f2b9175e-84eb-4f09-81e9-8c55a60c08e5 (ver = 1) >Thread-76::DEBUG::2014-01-23 15:32:32,740::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-76::DEBUG::2014-01-23 15:32:32,740::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-76::DEBUG::2014-01-23 15:32:32,753::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-76::DEBUG::2014-01-23 15:32:32,781::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-76::DEBUG::2014-01-23 15:32:32,854::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-76::DEBUG::2014-01-23 15:32:32,855::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-76::DEBUG::2014-01-23 15:32:32,865::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-76::DEBUG::2014-01-23 15:32:32,867::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-76::INFO::2014-01-23 15:32:32,867::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_imageNS already registered >Thread-76::INFO::2014-01-23 15:32:32,867::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_volumeNS already registered >Thread-76::DEBUG::2014-01-23 15:32:32,867::sp::1366::Storage.StoragePool::(setMasterDomain) Master domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 verified, version 1 >Thread-76::DEBUG::2014-01-23 15:32:32,888::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-76::INFO::2014-01-23 15:32:32,889::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-76::DEBUG::2014-01-23 15:32:32,889::sp::1395::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `def8e1c1-183d-4a22-82c0-483e982952f0` started monitoring domain `f2b9175e-84eb-4f09-81e9-8c55a60c08e5` >Thread-78::DEBUG::2014-01-23 15:32:32,889::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-78::DEBUG::2014-01-23 15:32:32,904::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-76::DEBUG::2014-01-23 15:32:32,909::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-76::DEBUG::2014-01-23 15:32:32,910::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-78::DEBUG::2014-01-23 15:32:32,915::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.000788807 s, 659 kB/s\n'; <rc> = 0 >Thread-78::DEBUG::2014-01-23 15:32:32,923::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 changed its status to Valid >Thread-79::DEBUG::2014-01-23 15:32:32,924::misc::884::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-79::DEBUG::2014-01-23 15:32:32,924::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-79::DEBUG::2014-01-23 15:32:32,924::misc::904::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-80::INFO::2014-01-23 15:32:32,927::clientIF::126::vds::(contEIOVms) vmContainerLock acquired >Thread-76::DEBUG::2014-01-23 15:32:32,937::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-76::INFO::2014-01-23 15:32:32,939::sp::1104::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 to /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-76::INFO::2014-01-23 15:32:32,939::sp::1104::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 to /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/mastersd >Thread-76::INFO::2014-01-23 15:32:32,940::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-82::DEBUG::2014-01-23 15:32:32,951::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-82::INFO::2014-01-23 15:32:32,951::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-82::DEBUG::2014-01-23 15:32:32,965::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-82::INFO::2014-01-23 15:32:32,965::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'hosted_datacenter', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'master_uuid': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'version': '3', 'spm_id': -1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'status': 'Active', 'diskfree': '262820331520', 'isoprefix': '', 'alerts': [], 'disktotal': '731155267584', 'version': 3}}} >Thread-78::DEBUG::2014-01-23 15:32:42,934::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-78::DEBUG::2014-01-23 15:32:42,943::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.000957186 s, 543 kB/s\n'; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:32:44,463::vdsm::55::vds::(sigtermHandler) Received signal 15 >MainThread::INFO::2014-01-23 15:32:44,963::vmChannels::183::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-01-23 15:32:44,964::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:32:44,964::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >MainThread::INFO::2014-01-23 15:32:45,115::domainMonitor::125::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::INFO::2014-01-23 15:32:45,116::domainMonitor::113::Storage.DomainMonitor::(stopMonitoring) Stop monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-78::DEBUG::2014-01-23 15:32:45,116::domainMonitor::175::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >VM Channels Listener::INFO::2014-01-23 15:32:45,340::vmChannels::178::vds::(run) VM channels listener thread has ended. >MainThread::INFO::2014-01-23 15:32:47,117::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::INFO::2014-01-23 15:32:47,118::vdsm::125::vds::(run) VDSM main thread ended. Waiting for 14 other threads... >MainThread::INFO::2014-01-23 15:32:47,118::vdsm::129::vds::(run) <Thread(libvirtEventLoop, started daemon 140339822692096)> >MainThread::INFO::2014-01-23 15:32:47,118::vdsm::129::vds::(run) <WorkerThread(Thread-6, started daemon 140339700545280)> >MainThread::INFO::2014-01-23 15:32:47,118::vdsm::129::vds::(run) <WorkerThread(6c7d9872-644d-448b-8f86-06a45d529f5d, started daemon 140339734116096)> >MainThread::INFO::2014-01-23 15:32:47,118::vdsm::129::vds::(run) <WorkerThread(Thread-8, started daemon 140339683759872)> >MainThread::INFO::2014-01-23 15:32:47,118::vdsm::129::vds::(run) <WorkerThread(Thread-1, started daemon 140339814299392)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <PolicyEngine(PolicyEngine, started daemon 140339079780096)> >MainThread::INFO::2014-01-23 15:32:47,119::momIF::84::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <MomThread(MOM, started 140339088172800)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <WorkerThread(Thread-9, started daemon 140339130136320)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <WorkerThread(Thread-5, started daemon 140339708937984)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <WorkerThread(Thread-10, started daemon 140339121743616)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <WorkerThread(Thread-7, started daemon 140339692152576)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <_MainThread(MainThread, started 140340034717504)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <WorkerThread(f6aeda9b-d28b-4bca-8608-50974cbfd006, started daemon 140339717330688)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <WorkerThread(Thread-3, started daemon 140339725723392)> >MainThread::INFO::2014-01-23 15:32:47,119::vdsm::129::vds::(run) <HostMonitor(HostMonitor, started daemon 140338719090432)> >MainThread::INFO::2014-01-23 15:32:53,750::vdsm::119::vds::(run) (PID: 5780) I am the actual vdsm 4.14.1-2.fc19 localhost.localdomain (3.12.8-200.fc19.x86_64) >MainThread::WARNING::2014-01-23 15:32:53,753::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-01-23 15:32:53,889::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-01-23 15:32:53,890::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) '/usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type' (cwd None) >MainThread::DEBUG::2014-01-23 15:32:53,909::hsm::423::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-01-23 15:32:53,910::hsm::447::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-01-23 15:32:53,912::hsm::479::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-01-23 15:32:53,912::hsm::480::Storage.HSM::(__cleanStorageRepository) Mount list: ['/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday'] >MainThread::DEBUG::2014-01-23 15:32:53,912::hsm::482::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-01-23 15:32:53,914::hsm::525::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-01-23 15:32:53,915::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-01-23 15:32:53,915::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >MainThread::INFO::2014-01-23 15:32:53,921::dispatcher::99::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >MainThread::INFO::2014-01-23 15:32:53,958::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x1a0add0>>) >MainThread::INFO::2014-01-23 15:32:53,958::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::INFO::2014-01-23 15:32:53,960::momIF::47::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-01-23 15:32:53,965::vmChannels::187::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-01-23 15:32:53,966::vmChannels::170::vds::(run) Starting VM channels listener thread. >clientIFinit::DEBUG::2014-01-23 15:32:53,976::libvirtconnection::145::root::(get) trying to connect libvirt >MainThread::WARNING::2014-01-23 15:32:53,984::clientIF::181::vds::(_prepareBindings) Unable to load the json rpc server module. Please make sure it is installed. >clientIFinit::DEBUG::2014-01-23 15:32:54,157::clientIF::518::vds::(_cleanOldFiles) removing old file 8086ca4f-c231-4955-8149-88133673bc7f.recovery >Thread-13::DEBUG::2014-01-23 15:32:54,602::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-13::DEBUG::2014-01-23 15:32:54,602::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Recovering from crash or Initializing', 'code': 99}} >Thread-14::DEBUG::2014-01-23 15:32:55,614::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-14::DEBUG::2014-01-23 15:32:55,614::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Recovering from crash or Initializing', 'code': 99}} >storageRefresh::DEBUG::2014-01-23 15:32:55,933::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:32:56,001::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:32:56,122::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm pvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:32:56,143::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:32:56,144::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:32:56,161::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:32:56,162::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:32:56,179::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-01-23 15:32:56,180::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >storageRefresh::DEBUG::2014-01-23 15:32:56,196::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >storageRefresh::WARNING::2014-01-23 15:32:56,197::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/hsm-tasks already exists >storageRefresh::DEBUG::2014-01-23 15:32:56,197::hsm::391::Storage.HSM::(storageRefresh) HSM is ready >Thread-15::DEBUG::2014-01-23 15:32:56,626::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call getHardwareInfo with () {} >Thread-15::DEBUG::2014-01-23 15:32:56,645::BindingXMLRPC::977::vds::(wrapper) return getHardwareInfo with {'status': {'message': 'Done', 'code': 0}, 'info': {'systemProductName': 'oVirt Node', 'systemSerialNumber': '4C4C4544-0044-5010-8038-B6C04F37354A', 'systemFamily': 'Not Specified', 'systemVersion': '18-1', 'systemUUID': '339190d0-b740-403f-9394-92c6882c4898', 'systemManufacturer': 'oVirt'}} >Thread-16::DEBUG::2014-01-23 15:32:56,655::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-16::INFO::2014-01-23 15:32:56,656::logUtils::44::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList(options=None) >Thread-16::INFO::2014-01-23 15:32:56,656::logUtils::47::dispatcher::(wrapper) Run and protect: getConnectedStoragePoolsList, Return response: {'poollist': []} >Thread-17::DEBUG::2014-01-23 15:32:56,666::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-17::INFO::2014-01-23 15:32:56,666::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'iqn': '', 'portal': '', 'user': 'kvm', 'protocol_version': '3', 'password': '******', 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}], options=None) >Thread-17::DEBUG::2014-01-23 15:32:56,669::hsm::2301::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday >Thread-17::DEBUG::2014-01-23 15:32:56,767::hsm::2325::Storage.HSM::(__prefetchDomains) Found SD uuids: ('f2b9175e-84eb-4f09-81e9-8c55a60c08e5',) >Thread-17::DEBUG::2014-01-23 15:32:56,767::hsm::2381::Storage.HSM::(connectStorageServer) knownSDs: {f2b9175e-84eb-4f09-81e9-8c55a60c08e5: storage.nfsSD.findDomain} >Thread-17::INFO::2014-01-23 15:32:56,767::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '48dd6100-83c5-4355-9c97-f1b22cf857dd'}]} >Thread-18::DEBUG::2014-01-23 15:32:56,779::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-18::INFO::2014-01-23 15:32:56,779::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', hostID=1, msdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', masterVersion=1, domainsMap=None, options=None) >Thread-18::INFO::2014-01-23 15:32:56,780::sp::133::Storage.StoragePool::(setBackend) updating pool def8e1c1-183d-4a22-82c0-483e982952f0 backend from type NoneType instance 0x3cf758bf00 to type StoragePoolDiskBackend instance 0x7f55700b67e8 >Thread-18::INFO::2014-01-23 15:32:56,781::sp::618::Storage.StoragePool::(connect) Connect host #1 to the storage pool def8e1c1-183d-4a22-82c0-483e982952f0 with master domain: f2b9175e-84eb-4f09-81e9-8c55a60c08e5 (ver = 1) >Thread-18::DEBUG::2014-01-23 15:32:56,781::iscsi::393::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-18::DEBUG::2014-01-23 15:32:56,781::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None) >Thread-18::DEBUG::2014-01-23 15:32:56,793::multipath::110::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath -r' (cwd None) >Thread-18::DEBUG::2014-01-23 15:32:56,825::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-01-23 15:32:56,930::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-18::DEBUG::2014-01-23 15:32:57,029::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-18::DEBUG::2014-01-23 15:32:57,039::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-18::DEBUG::2014-01-23 15:32:57,043::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-18::DEBUG::2014-01-23 15:32:57,043::sp::1366::Storage.StoragePool::(setMasterDomain) Master domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 verified, version 1 >Thread-18::DEBUG::2014-01-23 15:32:57,051::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-18::INFO::2014-01-23 15:32:57,051::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-20::DEBUG::2014-01-23 15:32:57,051::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-18::DEBUG::2014-01-23 15:32:57,051::sp::1395::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `def8e1c1-183d-4a22-82c0-483e982952f0` started monitoring domain `f2b9175e-84eb-4f09-81e9-8c55a60c08e5` >Thread-18::DEBUG::2014-01-23 15:32:57,060::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-18::DEBUG::2014-01-23 15:32:57,061::lvm::295::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name' (cwd None) >Thread-18::DEBUG::2014-01-23 15:32:57,083::lvm::295::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No device found for PV JukJuH-US8g-vpWK-AO8b-sC3i-PSpq-P4W50d.\n No volume groups found\n'; <rc> = 0 >Thread-18::INFO::2014-01-23 15:32:57,086::sp::1104::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 to /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-18::INFO::2014-01-23 15:32:57,086::sp::1104::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 to /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/mastersd >Thread-18::INFO::2014-01-23 15:32:57,087::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-22::DEBUG::2014-01-23 15:32:57,097::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-22::INFO::2014-01-23 15:32:57,098::logUtils::44::dispatcher::(wrapper) Run and protect: spmStart(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', prevID=-1, prevLVER=-1, maxHostID=250, domVersion=3, options=None) >Thread-22::INFO::2014-01-23 15:32:57,099::logUtils::47::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:32:57,112::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-23::DEBUG::2014-01-23 15:32:57,114::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-23::INFO::2014-01-23 15:32:57,114::logUtils::44::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-20::DEBUG::2014-01-23 15:32:57,191::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-20::DEBUG::2014-01-23 15:32:57,197::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.00076906 s, 676 kB/s\n'; <rc> = 0 >Thread-20::DEBUG::2014-01-23 15:33:18,145::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 changed its status to Valid >Thread-24::DEBUG::2014-01-23 15:33:18,146::misc::884::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-24::DEBUG::2014-01-23 15:33:18,146::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-24::DEBUG::2014-01-23 15:33:18,146::misc::904::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-25::INFO::2014-01-23 15:33:18,148::clientIF::126::vds::(contEIOVms) vmContainerLock acquired >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,248::sp::266::Storage.StoragePool::(startSpm) spm lock acquired successfully >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,256::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,256::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,256::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,256::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f3104774924c5313d393ba743b06d434808b2083'] >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,355::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >d4a87ae8-f452-4c22-bed9-a98c140ada22::INFO::2014-01-23 15:33:18,356::sp::431::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `f2b9175e-84eb-4f09-81e9-8c55a60c08e5` >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,360::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,361::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,368::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f3104774924c5313d393ba743b06d434808b2083'] >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,370::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >d4a87ae8-f452-4c22-bed9-a98c140ada22::INFO::2014-01-23 15:33:18,370::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_imageNS already registered >d4a87ae8-f452-4c22-bed9-a98c140ada22::INFO::2014-01-23 15:33:18,370::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace f2b9175e-84eb-4f09-81e9-8c55a60c08e5_volumeNS already registered >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,370::sp::436::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,377::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f3104774924c5313d393ba743b06d434808b2083'] >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,377::sp::443::Storage.StoragePool::(_upgradePool) Registering with state change event >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,377::sp::446::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,378::sd::623::Storage.StorageDomain::(createMasterTree) creating vms dir: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/master/vms >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,378::sd::626::Storage.StorageDomain::(createMasterTree) creating task dir: /rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/master/tasks >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,385::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f3104774924c5313d393ba743b06d434808b2083'] >d4a87ae8-f452-4c22-bed9-a98c140ada22::DEBUG::2014-01-23 15:33:18,386::sp::326::Storage.StoragePool::(startSpm) ended. >Thread-23::INFO::2014-01-23 15:33:18,388::sp::1015::Storage.StoragePool::(activateSD) sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 spUUID=def8e1c1-183d-4a22-82c0-483e982952f0 >Thread-23::INFO::2014-01-23 15:33:18,388::logUtils::47::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-27::DEBUG::2014-01-23 15:33:18,398::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-27::INFO::2014-01-23 15:33:18,399::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-27::INFO::2014-01-23 15:33:18,399::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-28::DEBUG::2014-01-23 15:33:18,408::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-28::INFO::2014-01-23 15:33:18,408::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-28::INFO::2014-01-23 15:33:18,445::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-29::DEBUG::2014-01-23 15:33:18,455::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-29::INFO::2014-01-23 15:33:18,455::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-29::INFO::2014-01-23 15:33:18,459::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'hosted_datacenter', 'isoprefix': '', 'pool_status': 'connected', 'lver': 0, 'domains': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'master_uuid': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'status': 'Active', 'diskfree': '262820331520', 'isoprefix': '', 'alerts': [], 'disktotal': '731155267584', 'version': 3}}} >Thread-30::DEBUG::2014-01-23 15:33:18,469::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-30::INFO::2014-01-23 15:33:18,470::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-30::INFO::2014-01-23 15:33:18,470::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00076906', 'lastCheck': '0.3', 'valid': True}} >Thread-31::DEBUG::2014-01-23 15:33:18,499::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-31::INFO::2014-01-23 15:33:18,499::logUtils::44::dispatcher::(wrapper) Run and protect: createVolume(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', imgUUID='d3ba7005-ddf4-4016-ab02-32790a5203d3', size=52428800, volFormat=5, preallocate=2, diskType=2, volUUID='5e499db1-444a-43a3-ba15-a16c382d7359', desc='Hosted Engine Image', srcImgUUID='00000000-0000-0000-0000-000000000000', srcVolUUID='00000000-0000-0000-0000-000000000000') >Thread-31::INFO::2014-01-23 15:33:18,598::logUtils::47::dispatcher::(wrapper) Run and protect: createVolume, Return response: None >Thread-32::DEBUG::2014-01-23 15:33:18,818::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-32::INFO::2014-01-23 15:33:18,819::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-32::INFO::2014-01-23 15:33:18,819::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'85e33216-2c48-4c78-88ee-d3ec8bbe2082': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '85e33216-2c48-4c78-88ee-d3ec8bbe2082'}}} >85e33216-2c48-4c78-88ee-d3ec8bbe2082::DEBUG::2014-01-23 15:33:18,940::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for 5e499db1-444a-43a3-ba15-a16c382d7359 >85e33216-2c48-4c78-88ee-d3ec8bbe2082::DEBUG::2014-01-23 15:33:18,947::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for 5e499db1-444a-43a3-ba15-a16c382d7359 >85e33216-2c48-4c78-88ee-d3ec8bbe2082::DEBUG::2014-01-23 15:33:18,954::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for 5e499db1-444a-43a3-ba15-a16c382d7359 >85e33216-2c48-4c78-88ee-d3ec8bbe2082::INFO::2014-01-23 15:33:18,967::image::215::Storage.Image::(getChain) sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 imgUUID=d3ba7005-ddf4-4016-ab02-32790a5203d3 chain=[<storage.fileVolume.FileVolume object at 0x7f55700860d0>] >85e33216-2c48-4c78-88ee-d3ec8bbe2082::INFO::2014-01-23 15:33:18,974::volume::443::Storage.Volume::(create) Creating volume 5e499db1-444a-43a3-ba15-a16c382d7359 >85e33216-2c48-4c78-88ee-d3ec8bbe2082::ERROR::2014-01-23 15:33:19,297::volume::466::Storage.Volume::(create) Failed to create volume: /rhev/data-center/def8e1c1-183d-4a22-82c0-483e982952f0/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359, volume already exists >85e33216-2c48-4c78-88ee-d3ec8bbe2082::ERROR::2014-01-23 15:33:19,297::volume::502::Storage.Volume::(create) Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/volume.py", line 468, in create > raise e >VolumeAlreadyExists: Volume already exists: ('5e499db1-444a-43a3-ba15-a16c382d7359',) >85e33216-2c48-4c78-88ee-d3ec8bbe2082::INFO::2014-01-23 15:33:19,742::volume::346::Storage.Volume::(startCreateVolumeRollback) startCreateVolumeRollback: sdUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5 imgUUID=d3ba7005-ddf4-4016-ab02-32790a5203d3 volUUID=5e499db1-444a-43a3-ba15-a16c382d7359 >Thread-33::DEBUG::2014-01-23 15:33:19,830::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-33::INFO::2014-01-23 15:33:19,831::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-33::INFO::2014-01-23 15:33:19,831::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {'85e33216-2c48-4c78-88ee-d3ec8bbe2082': {'code': 212, 'message': 'Volume already exists', 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': '85e33216-2c48-4c78-88ee-d3ec8bbe2082'}}} >Thread-34::DEBUG::2014-01-23 15:33:19,841::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-34::INFO::2014-01-23 15:33:19,841::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID='85e33216-2c48-4c78-88ee-d3ec8bbe2082', spUUID=None, options=None) >Thread-34::INFO::2014-01-23 15:33:19,968::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-35::DEBUG::2014-01-23 15:33:19,979::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-35::INFO::2014-01-23 15:33:19,979::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-35::INFO::2014-01-23 15:33:19,979::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-36::DEBUG::2014-01-23 15:33:19,988::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-36::INFO::2014-01-23 15:33:19,989::logUtils::44::dispatcher::(wrapper) Run and protect: spmStop(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', options=None) >Thread-36::DEBUG::2014-01-23 15:33:19,990::sp::336::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process >Thread-36::DEBUG::2014-01-23 15:33:19,991::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-36::DEBUG::2014-01-23 15:33:19,993::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-36::DEBUG::2014-01-23 15:33:19,994::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-36::DEBUG::2014-01-23 15:33:20,030::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-36::INFO::2014-01-23 15:33:20,041::logUtils::47::dispatcher::(wrapper) Run and protect: spmStop, Return response: None >Thread-37::DEBUG::2014-01-23 15:33:20,051::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-37::INFO::2014-01-23 15:33:20,051::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID='def8e1c1-183d-4a22-82c0-483e982952f0', hostID=1, remove=False, options=None) >Thread-37::INFO::2014-01-23 15:33:20,053::sp::644::Storage.StoragePool::(disconnect) Disconnect from the storage pool def8e1c1-183d-4a22-82c0-483e982952f0 >Thread-37::DEBUG::2014-01-23 15:33:20,076::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=hosted_storage', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=ON', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=hosted_datacenter', 'POOL_DOMAINS=f2b9175e-84eb-4f09-81e9-8c55a60c08e5:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=def8e1c1-183d-4a22-82c0-483e982952f0', 'REMOTE_PATH=multipass.eng.lab.tlv.redhat.com:/export/images/rnd/mikekol/testday', 'ROLE=Master', 'SDUUID=f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=6f5dedddb409f285123aa4191c705d16bd01eeb6'] >Thread-37::INFO::2014-01-23 15:33:20,076::domainMonitor::113::Storage.DomainMonitor::(stopMonitoring) Stop monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-20::DEBUG::2014-01-23 15:33:20,076::domainMonitor::175::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-37::INFO::2014-01-23 15:33:22,077::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: True >Thread-38::DEBUG::2014-01-23 15:33:22,180::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-38::INFO::2014-01-23 15:33:22,180::logUtils::44::dispatcher::(wrapper) Run and protect: startMonitoringDomain(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', hostID=1, options=None) >Thread-38::INFO::2014-01-23 15:33:22,181::domainMonitor::99::Storage.DomainMonitor::(startMonitoring) Start monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-39::DEBUG::2014-01-23 15:33:22,181::domainMonitor::165::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-38::INFO::2014-01-23 15:33:22,182::logUtils::47::dispatcher::(wrapper) Run and protect: startMonitoringDomain, Return response: None >Thread-39::DEBUG::2014-01-23 15:33:22,191::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-39::DEBUG::2014-01-23 15:33:22,196::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.000384106 s, 1.4 MB/s\n'; <rc> = 0 >Thread-39::DEBUG::2014-01-23 15:33:22,198::domainMonitor::247::Storage.DomainMonitorThread::(_monitorDomain) Domain f2b9175e-84eb-4f09-81e9-8c55a60c08e5 changed its status to Valid >Thread-40::DEBUG::2014-01-23 15:33:22,198::misc::884::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-40::DEBUG::2014-01-23 15:33:22,198::misc::894::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-40::DEBUG::2014-01-23 15:33:22,199::misc::904::Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-41::INFO::2014-01-23 15:33:22,200::clientIF::126::vds::(contEIOVms) vmContainerLock acquired >Thread-42::INFO::2014-01-23 15:33:27,195::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-42::INFO::2014-01-23 15:33:27,286::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000384106', 'lastCheck': '5.1', 'valid': True}} >Thread-42::ERROR::2014-01-23 15:33:27,287::brokerlink::72::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(connect) Failed to connect to broker: [Errno 2] No such file or directory >Thread-42::ERROR::2014-01-23 15:33:27,287::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 202, in get_local_host_score > with broker.connection(): > File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ > return self.gen.next() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 96, in connection > self.connect() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 64, in connect > self._socket.connect(constants.BROKER_SOCKET_FILE) > File "/usr/lib64/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) >error: [Errno 2] No such file or directory >Thread-39::DEBUG::2014-01-23 15:33:32,214::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-39::DEBUG::2014-01-23 15:33:32,241::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.00056112 s, 927 kB/s\n'; <rc> = 0 >Thread-43::INFO::2014-01-23 15:33:32,373::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-43::INFO::2014-01-23 15:33:32,373::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00056112', 'lastCheck': '0.1', 'valid': True}} >Thread-43::ERROR::2014-01-23 15:33:32,374::brokerlink::72::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(connect) Failed to connect to broker: [Errno 2] No such file or directory >Thread-43::ERROR::2014-01-23 15:33:32,375::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 202, in get_local_host_score > with broker.connection(): > File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ > return self.gen.next() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 96, in connection > self.connect() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 64, in connect > self._socket.connect(constants.BROKER_SOCKET_FILE) > File "/usr/lib64/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) >error: [Errno 2] No such file or directory >Thread-44::INFO::2014-01-23 15:33:37,394::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-44::INFO::2014-01-23 15:33:37,403::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00056112', 'lastCheck': '5.2', 'valid': True}} >Thread-44::ERROR::2014-01-23 15:33:37,404::brokerlink::72::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(connect) Failed to connect to broker: [Errno 2] No such file or directory >Thread-44::ERROR::2014-01-23 15:33:37,404::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 202, in get_local_host_score > with broker.connection(): > File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ > return self.gen.next() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 96, in connection > self.connect() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 64, in connect > self._socket.connect(constants.BROKER_SOCKET_FILE) > File "/usr/lib64/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) >error: [Errno 2] No such file or directory >Thread-39::DEBUG::2014-01-23 15:33:42,253::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-39::DEBUG::2014-01-23 15:33:42,259::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.000823172 s, 632 kB/s\n'; <rc> = 0 >Thread-45::INFO::2014-01-23 15:33:42,423::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-45::INFO::2014-01-23 15:33:42,459::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000823172', 'lastCheck': '0.2', 'valid': True}} >Thread-45::ERROR::2014-01-23 15:33:42,460::brokerlink::72::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(connect) Failed to connect to broker: [Errno 2] No such file or directory >Thread-45::ERROR::2014-01-23 15:33:42,461::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 202, in get_local_host_score > with broker.connection(): > File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ > return self.gen.next() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 96, in connection > self.connect() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 64, in connect > self._socket.connect(constants.BROKER_SOCKET_FILE) > File "/usr/lib64/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) >error: [Errno 2] No such file or directory >Thread-46::INFO::2014-01-23 15:33:47,489::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-46::INFO::2014-01-23 15:33:47,489::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000823172', 'lastCheck': '5.2', 'valid': True}} >Thread-46::ERROR::2014-01-23 15:33:47,490::brokerlink::72::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(connect) Failed to connect to broker: [Errno 2] No such file or directory >Thread-46::ERROR::2014-01-23 15:33:47,491::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 202, in get_local_host_score > with broker.connection(): > File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ > return self.gen.next() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 96, in connection > self.connect() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 64, in connect > self._socket.connect(constants.BROKER_SOCKET_FILE) > File "/usr/lib64/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) >error: [Errno 2] No such file or directory >Thread-39::DEBUG::2014-01-23 15:33:52,272::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-39::DEBUG::2014-01-23 15:33:52,279::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.00109192 s, 476 kB/s\n'; <rc> = 0 >Thread-47::INFO::2014-01-23 15:33:52,510::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-47::INFO::2014-01-23 15:33:52,530::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'f2b9175e-84eb-4f09-81e9-8c55a60c08e5': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00109192', 'lastCheck': '0.2', 'valid': True}} >Thread-47::ERROR::2014-01-23 15:33:52,531::brokerlink::72::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(connect) Failed to connect to broker: [Errno 2] No such file or directory >Thread-47::ERROR::2014-01-23 15:33:52,531::API::1244::vds::(getStats) failed to retrieve Hosted Engine HA score >Traceback (most recent call last): > File "/usr/share/vdsm/API.py", line 1242, in getStats > stats['haScore'] = haClient.HAClient().get_local_host_score() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 202, in get_local_host_score > with broker.connection(): > File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ > return self.gen.next() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 96, in connection > self.connect() > File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 64, in connect > self._socket.connect(constants.BROKER_SOCKET_FILE) > File "/usr/lib64/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) >error: [Errno 2] No such file or directory >Thread-48::DEBUG::2014-01-23 15:33:58,478::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmGetStats with ('8086ca4f-c231-4955-8149-88133673bc7f',) {} >Thread-48::DEBUG::2014-01-23 15:33:58,479::BindingXMLRPC::977::vds::(wrapper) return vmGetStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}} >Thread-49::DEBUG::2014-01-23 15:33:58,603::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmCreate with ({'emulatedMachine': 'pc', 'vmId': '8086ca4f-c231-4955-8149-88133673bc7f', 'devices': [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': '84bee1ea-913a-4fb8-84ca-11c4496e5737', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'format': 'raw', 'optional': 'false', 'poolID': '00000000-0000-0000-0000-000000000000', 'volumeID': '5e499db1-444a-43a3-ba15-a16c382d7359', 'imageID': 'd3ba7005-ddf4-4016-ab02-32790a5203d3', 'specParams': {}, 'readonly': 'false', 'domainID': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'deviceId': 'd3ba7005-ddf4-4016-ab02-32790a5203d3', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:74:b6:e0', 'linkActive': 'true', 'network': 'ovirtmgmt', 'bootOrder': '1', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': '74abceb2-5e28-4449-832f-64c297e3364c', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '61b83184-6d26-4c52-8fad-fa50168c579c', 'alias': 'console0'}], 'smp': '2', 'memSize': '4096', 'cpuType': 'Penryn', 'spiceSecureChannels': 'smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir', 'vmName': 'HostedEngine', 'display': 'vnc'},) {} >Thread-49::INFO::2014-01-23 15:33:58,608::API::668::vds::(_getNetworkIp) network None: using 0 >Thread-49::INFO::2014-01-23 15:33:58,608::clientIF::361::vds::(createVm) vmContainerLock acquired by vm 8086ca4f-c231-4955-8149-88133673bc7f >Thread-49::DEBUG::2014-01-23 15:33:58,609::libvirtconnection::145::root::(get) trying to connect libvirt >Thread-50::DEBUG::2014-01-23 15:33:58,630::vm::2226::vm.Vm::(_startUnderlyingVm) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::Start >Thread-49::DEBUG::2014-01-23 15:33:58,630::clientIF::374::vds::(createVm) Total desktops after creation of 8086ca4f-c231-4955-8149-88133673bc7f is 1 >Thread-50::DEBUG::2014-01-23 15:33:58,630::vm::2230::vm.Vm::(_startUnderlyingVm) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::_ongoingCreations acquired >Thread-49::DEBUG::2014-01-23 15:33:58,630::BindingXMLRPC::977::vds::(wrapper) return vmCreate with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'nicModel': 'rtl8139,pv', 'vmId': '8086ca4f-c231-4955-8149-88133673bc7f', 'emulatedMachine': 'pc', 'displayIp': '0', 'pid': '0', 'devices': [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': '84bee1ea-913a-4fb8-84ca-11c4496e5737', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'format': 'raw', 'optional': 'false', 'poolID': '00000000-0000-0000-0000-000000000000', 'volumeID': '5e499db1-444a-43a3-ba15-a16c382d7359', 'imageID': 'd3ba7005-ddf4-4016-ab02-32790a5203d3', 'specParams': {}, 'readonly': 'false', 'domainID': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'deviceId': 'd3ba7005-ddf4-4016-ab02-32790a5203d3', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:74:b6:e0', 'linkActive': 'true', 'network': 'ovirtmgmt', 'bootOrder': '1', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': '74abceb2-5e28-4449-832f-64c297e3364c', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '61b83184-6d26-4c52-8fad-fa50168c579c', 'alias': 'console0'}], 'smp': '2', 'vmType': 'kvm', 'displaySecurePort': '-1', 'memSize': '4096', 'displayPort': '-1', 'cpuType': 'Penryn', 'spiceSecureChannels': 'smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir', 'clientIp': '', 'vmName': 'HostedEngine', 'display': 'vnc'}} >Thread-50::INFO::2014-01-23 15:33:58,631::vm::3091::vm.Vm::(_run) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::VM wrapper has started >Thread-50::INFO::2014-01-23 15:33:58,632::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', spUUID='00000000-0000-0000-0000-000000000000', imgUUID='d3ba7005-ddf4-4016-ab02-32790a5203d3', volUUID='5e499db1-444a-43a3-ba15-a16c382d7359', options=None) >Thread-50::INFO::2014-01-23 15:33:58,638::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '512', 'apparentsize': '26843545600'} >Thread-50::INFO::2014-01-23 15:33:58,638::clientIF::320::vds::(prepareVolumePath) prepared volume path: >Thread-50::INFO::2014-01-23 15:33:58,638::logUtils::44::dispatcher::(wrapper) Run and protect: prepareImage(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', spUUID='00000000-0000-0000-0000-000000000000', imgUUID='d3ba7005-ddf4-4016-ab02-32790a5203d3', leafUUID='5e499db1-444a-43a3-ba15-a16c382d7359') >Thread-50::WARNING::2014-01-23 15:33:58,641::fileUtils::167::Storage.fileUtils::(createdir) Dir /var/run/vdsm/storage/f2b9175e-84eb-4f09-81e9-8c55a60c08e5 already exists >Thread-50::DEBUG::2014-01-23 15:33:58,641::fileSD::446::Storage.StorageDomain::(createImageLinks) img run dir already exists: /var/run/vdsm/storage/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/d3ba7005-ddf4-4016-ab02-32790a5203d3 >Thread-50::DEBUG::2014-01-23 15:33:58,642::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for 5e499db1-444a-43a3-ba15-a16c382d7359 >Thread-50::INFO::2014-01-23 15:33:58,646::logUtils::47::dispatcher::(wrapper) Run and protect: prepareImage, Return response: {'info': {'domainID': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'volType': 'path', 'leaseOffset': 0, 'path': '/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359', 'volumeID': '5e499db1-444a-43a3-ba15-a16c382d7359', 'leasePath': '/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359.lease', 'imageID': 'd3ba7005-ddf4-4016-ab02-32790a5203d3'}, 'path': '/var/run/vdsm/storage/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359', 'imgVolumesInfo': [{'domainID': 'f2b9175e-84eb-4f09-81e9-8c55a60c08e5', 'volType': 'path', 'leaseOffset': 0, 'path': '/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359', 'volumeID': '5e499db1-444a-43a3-ba15-a16c382d7359', 'leasePath': '/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359.lease', 'imageID': 'd3ba7005-ddf4-4016-ab02-32790a5203d3'}]} >Thread-50::INFO::2014-01-23 15:33:58,646::clientIF::320::vds::(prepareVolumePath) prepared volume path: /var/run/vdsm/storage/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359 >Thread-50::DEBUG::2014-01-23 15:33:58,683::utils::555::root::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_hostedengine' (cwd None) >Thread-50::DEBUG::2014-01-23 15:33:58,737::utils::575::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::INFO::2014-01-23 15:33:58,737::hooks::100::root::(_runHooksDir) >Thread-50::DEBUG::2014-01-23 15:33:58,738::vm::3130::vm.Vm::(_run) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::<?xml version="1.0" encoding="utf-8"?><domain type="kvm"> > <name>HostedEngine</name> > <uuid>8086ca4f-c231-4955-8149-88133673bc7f</uuid> > <memory>4194304</memory> > <currentMemory>4194304</currentMemory> > <vcpu current="2">2</vcpu> > <memtune> > <min_guarantee>0</min_guarantee> > </memtune> > <devices> > <channel type="unix"> > <target name="com.redhat.rhevm.vdsm" type="virtio"/> > <source mode="bind" path="/var/lib/libvirt/qemu/channels/8086ca4f-c231-4955-8149-88133673bc7f.com.redhat.rhevm.vdsm"/> > </channel> > <channel type="unix"> > <target name="org.qemu.guest_agent.0" type="virtio"/> > <source mode="bind" path="/var/lib/libvirt/qemu/channels/8086ca4f-c231-4955-8149-88133673bc7f.org.qemu.guest_agent.0"/> > </channel> > <input bus="ps2" type="mouse"/> > <graphics autoport="yes" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" type="vnc"/> > <console type="pty"> > <target port="0" type="virtio"/> > </console> > <memballoon model="none"/> > <controller model="virtio-scsi" type="scsi"/> > <interface type="bridge"> > <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci"/> > <mac address="00:16:3e:74:b6:e0"/> > <model type="virtio"/> > <source bridge="ovirtmgmt"/> > <filterref filter="vdsm-no-mac-spoofing"/> > <link state="up"/> > <boot order="1"/> > </interface> > <disk device="cdrom" snapshot="no" type="file"> > <address bus="1" controller="0" target="0" type="drive" unit="0"/> > <source file="" startupPolicy="optional"/> > <target bus="ide" dev="hdc"/> > <readonly/> > <serial/> > </disk> > <disk device="disk" snapshot="no" type="file"> > <address bus="0x00" domain="0x0000" function="0x0" slot="0x06" type="pci"/> > <source file="/var/run/vdsm/storage/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359"/> > <target bus="virtio" dev="vda"/> > <serial>d3ba7005-ddf4-4016-ab02-32790a5203d3</serial> > <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> > </disk> > <lease> > <key>5e499db1-444a-43a3-ba15-a16c382d7359</key> > <lockspace>f2b9175e-84eb-4f09-81e9-8c55a60c08e5</lockspace> > <target offset="0" path="/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/images/d3ba7005-ddf4-4016-ab02-32790a5203d3/5e499db1-444a-43a3-ba15-a16c382d7359.lease"/> > </lease> > </devices> > <os> > <type arch="x86_64" machine="pc">hvm</type> > <smbios mode="sysinfo"/> > </os> > <sysinfo type="smbios"> > <system> > <entry name="manufacturer">oVirt</entry> > <entry name="product">oVirt Node</entry> > <entry name="version">19-4</entry> > <entry name="serial">339190D0-B740-403F-9394-92C6882C4898_66:1a:4a:23:11:6c</entry> > <entry name="uuid">8086ca4f-c231-4955-8149-88133673bc7f</entry> > </system> > </sysinfo> > <clock adjustment="0" offset="variable"> > <timer name="rtc" tickpolicy="catchup"/> > </clock> > <features> > <acpi/> > </features> > <cpu match="exact"> > <model>Penryn</model> > </cpu> ><on_poweroff>destroy</on_poweroff><on_reboot>destroy</on_reboot><on_crash>destroy</on_crash></domain> >Thread-52::DEBUG::2014-01-23 15:33:58,751::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-52::ERROR::2014-01-23 15:33:58,752::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-53::DEBUG::2014-01-23 15:34:00,021::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-53::ERROR::2014-01-23 15:34:00,021::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-54::DEBUG::2014-01-23 15:34:01,129::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-54::ERROR::2014-01-23 15:34:01,129::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-55::DEBUG::2014-01-23 15:34:02,237::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-55::ERROR::2014-01-23 15:34:02,237::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-39::DEBUG::2014-01-23 15:34:02,292::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/usr/bin/dd iflag=direct if=/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_mikekol_testday/f2b9175e-84eb-4f09-81e9-8c55a60c08e5/dom_md/metadata bs=4096 count=1' (cwd None) >Thread-39::DEBUG::2014-01-23 15:34:02,321::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n520 bytes (520 B) copied, 0.00101465 s, 512 kB/s\n'; <rc> = 0 >Thread-56::DEBUG::2014-01-23 15:34:03,343::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-56::ERROR::2014-01-23 15:34:03,343::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-50::DEBUG::2014-01-23 15:34:04,390::libvirtconnection::124::root::(wrapper) Unknown libvirterror: ecode: 63 edom: 33 level: 2 message: Error while building firewall: Some rules could not be created for interface vnet0: Failure to execute command '$EBT -t nat -N libvirt-J-vnet0' : '[91mFirewallD is not running[00m'. > >Thread-50::DEBUG::2014-01-23 15:34:04,391::vm::2247::vm.Vm::(_startUnderlyingVm) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::_ongoingCreations released >Thread-50::ERROR::2014-01-23 15:34:04,391::vm::2273::vm.Vm::(_startUnderlyingVm) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::The vm start process failed >Traceback (most recent call last): > File "/usr/share/vdsm/vm.py", line 2233, in _startUnderlyingVm > self._run() > File "/usr/share/vdsm/vm.py", line 3164, in _run > self._connection.createXML(domxml, flags), > File "/usr/lib64/python2.7/site-packages/vdsm/libvirtconnection.py", line 92, in wrapper > ret = f(*args, **kwargs) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2805, in createXML > if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) >libvirtError: Error while building firewall: Some rules could not be created for interface vnet0: Failure to execute command '$EBT -t nat -N libvirt-J-vnet0' : '[91mFirewallD is not running[00m'. > >Thread-50::DEBUG::2014-01-23 15:34:04,394::vm::2725::vm.Vm::(setDownStatus) vmId=`8086ca4f-c231-4955-8149-88133673bc7f`::Changed state to Down: Error while building firewall: Some rules could not be created for interface vnet0: Failure to execute command '$EBT -t nat -N libvirt-J-vnet0' : '[91mFirewallD is not running[00m'. > >Thread-57::DEBUG::2014-01-23 15:34:04,481::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-57::ERROR::2014-01-23 15:34:04,518::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-58::DEBUG::2014-01-23 15:34:05,620::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-58::ERROR::2014-01-23 15:34:05,620::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-59::DEBUG::2014-01-23 15:34:06,719::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-59::ERROR::2014-01-23 15:34:06,720::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-60::DEBUG::2014-01-23 15:34:07,821::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-60::ERROR::2014-01-23 15:34:07,822::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-61::DEBUG::2014-01-23 15:34:08,922::BindingXMLRPC::970::vds::(wrapper) client [127.0.0.1]::call vmSetTicket with ('8086ca4f-c231-4955-8149-88133673bc7f', '5381JNQz', '10800', 'disconnect', {}) {} >Thread-61::ERROR::2014-01-23 15:34:08,922::BindingXMLRPC::989::vds::(wrapper) unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/BindingXMLRPC.py", line 222, in vmSetTicket > return vm.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/API.py", line 611, in setTicket > return v.setTicket(password, ttl, existingConnAction, params) > File "/usr/share/vdsm/vm.py", line 4466, in setTicket > graphics = _domParseStr(self._dom.XMLDesc(0)).childNodes[0]. \ >AttributeError: 'NoneType' object has no attribute 'XMLDesc' >Thread-62::DEBUG::2014-01-23 15:34:09,961::BindingXMLRPC::159::vds::(wrapper) client [127.0.0.1] >Thread-62::INFO::2014-01-23 15:34:09,962::logUtils::44::dispatcher::(wrapper) Run and protect: stopMonitoringDomain(sdUUID='f2b9175e-84eb-4f09-81e9-8c55a60c08e5', options=None) >Thread-62::INFO::2014-01-23 15:34:09,963::domainMonitor::113::Storage.DomainMonitor::(stopMonitoring) Stop monitoring f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-39::DEBUG::2014-01-23 15:34:09,963::domainMonitor::175::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for f2b9175e-84eb-4f09-81e9-8c55a60c08e5 >Thread-62::INFO::2014-01-23 15:34:11,964::logUtils::47::dispatcher::(wrapper) Run and protect: stopMonitoringDomain, Return response: None
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1057139
: 854405 |
854406