Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 827293 Details for
Bug 1026377
HA agent dies after manual migration of engine vm.
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
agen log
agent.log (text/x-log), 3.82 MB, created by
Artyom
on 2013-11-21 15:34:13 UTC
(
hide
)
Description:
agen log
Filename:
MIME Type:
Creator:
Artyom
Created:
2013-11-21 15:34:13 UTC
Size:
3.82 MB
patch
obsolete
>MainThread::INFO::2013-11-14 14:15:58,632::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:15:58,696::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:15:58,755::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:15:58,755::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:15:58,756::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:15:58,756::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:15:58,756::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:15:58,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:15:58,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.375 >MainThread::INFO::2013-11-14 14:15:58,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:15:58,757::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431358|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431358 (Thu Nov 14 14:15:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.375 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:15:58,758::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:15:58,766::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:15:58,766::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:15:58,767::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431358, 'last-update-local-ts': 1384431358.7663629, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:15:58,767::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:15:58,767::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431355, 'last-update-local-ts': 1384431358.7663629, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:15:58,767::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:16:08,857::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:16:09,105::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:16:09,202::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:16:09,263::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:16:09,264::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:16:09,264::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:16:09,264::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:09,265::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:16:09,265::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:16:09,265::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 14:16:09,265::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:09,266::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431369|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431369 (Thu Nov 14 14:16:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.32 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:16:09,267::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:16:09,275::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:16:09,276::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:16:09,276::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431369, 'last-update-local-ts': 1384431369.2756021, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:09,276::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:16:09,276::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431365, 'last-update-local-ts': 1384431369.2756021, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:09,276::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:16:19,361::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:16:19,611::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:16:19,703::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:16:19,766::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:16:19,766::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:16:19,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:16:19,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:19,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:16:19,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:16:19,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.27 >MainThread::INFO::2013-11-14 14:16:19,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:19,775::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431379|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431379 (Thu Nov 14 14:16:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.27 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:16:19,775::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:16:19,783::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:16:19,784::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:16:19,784::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431379, 'last-update-local-ts': 1384431379.7835741, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:19,784::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:16:19,784::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431376, 'last-update-local-ts': 1384431379.7835741, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:19,784::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:16:29,870::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:16:30,120::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:16:30,201::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:16:30,261::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:16:30,262::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:16:30,262::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:16:30,262::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:30,263::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:16:30,263::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:16:30,263::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 14:16:30,263::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:30,264::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431390|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431390 (Thu Nov 14 14:16:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:16:30,265::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:16:30,274::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:16:30,274::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:16:30,274::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431390, 'last-update-local-ts': 1384431390.274267, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:30,275::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:16:30,275::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431386, 'last-update-local-ts': 1384431390.274267, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:30,275::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:16:40,341::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:16:40,590::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:16:40,649::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:16:40,716::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:16:40,717::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:16:40,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:16:40,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:40,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:16:40,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:16:40,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 14:16:40,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:40,719::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431400|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431400 (Thu Nov 14 14:16:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:16:40,719::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:16:40,726::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:16:40,726::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:16:40,726::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431400, 'last-update-local-ts': 1384431400.7261801, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:40,727::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:16:40,727::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431397, 'last-update-local-ts': 1384431400.7261801, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:40,727::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:16:50,808::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:16:51,056::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:16:51,151::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:16:51,211::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:16:51,212::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:16:51,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:16:51,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:51,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:16:51,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:16:51,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 14:16:51,220::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:16:51,220::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431411|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431411 (Thu Nov 14 14:16:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:16:51,220::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:16:51,230::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:16:51,230::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:16:51,231::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431411, 'last-update-local-ts': 1384431411.230376, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:51,231::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:16:51,231::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431407, 'last-update-local-ts': 1384431411.230376, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:16:51,231::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:17:01,317::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:17:01,568::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:17:01,666::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:17:01,727::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:17:01,728::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:17:01,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:17:01,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:01,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:17:01,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:17:01,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 14:17:01,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:01,730::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431421|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431421 (Thu Nov 14 14:17:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:17:01,731::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:17:01,739::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:17:01,739::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:17:01,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431421, 'last-update-local-ts': 1384431421.7394071, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:01,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:17:01,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431418, 'last-update-local-ts': 1384431421.7394071, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:01,740::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:17:11,816::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:17:12,071::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:17:12,160::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:17:12,249::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:17:12,249::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:17:12,250::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:17:12,251::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:12,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:17:12,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:17:12,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 14:17:12,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:12,253::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431432|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431432 (Thu Nov 14 14:17:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:17:12,253::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:17:12,261::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:17:12,262::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:17:12,262::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431432, 'last-update-local-ts': 1384431432.2616839, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:12,262::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:17:12,262::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431428, 'last-update-local-ts': 1384431432.2616839, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:12,262::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:17:22,353::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:17:22,665::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:17:22,741::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:17:22,800::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:17:22,800::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:17:22,801::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:17:22,801::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:22,801::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:17:22,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:17:22,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 14:17:22,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:22,803::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431442|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431442 (Thu Nov 14 14:17:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:17:22,804::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:17:22,813::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:17:22,813::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:17:22,814::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431442, 'last-update-local-ts': 1384431442.8134601, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:22,814::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:17:22,814::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431439, 'last-update-local-ts': 1384431442.8134601, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:22,814::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:17:32,892::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:17:33,164::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:17:33,225::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:17:33,286::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:17:33,286::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:17:33,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:17:33,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:33,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:17:33,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:17:33,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.325 >MainThread::INFO::2013-11-14 14:17:33,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:33,288::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431453|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431453 (Thu Nov 14 14:17:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.325 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:17:33,289::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:17:33,297::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:17:33,297::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:17:33,298::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431453, 'last-update-local-ts': 1384431453.297343, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:33,298::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:17:33,298::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431449, 'last-update-local-ts': 1384431453.297343, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:33,298::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:17:43,381::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:17:43,641::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:17:43,731::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:17:43,791::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:17:43,791::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:17:43,798::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:17:43,798::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:43,798::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:17:43,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:17:43,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.315 >MainThread::INFO::2013-11-14 14:17:43,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:43,800::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431463|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431463 (Thu Nov 14 14:17:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.315 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:17:43,800::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:17:43,810::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:17:43,810::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:17:43,810::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431463, 'last-update-local-ts': 1384431463.8098929, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:43,810::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:17:43,810::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431460, 'last-update-local-ts': 1384431463.8098929, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:43,811::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:17:53,891::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:17:54,150::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:17:54,234::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:17:54,295::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:17:54,295::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:17:54,296::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:17:54,296::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:54,296::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:17:54,297::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:17:54,297::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 14:17:54,297::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:17:54,298::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431474|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431474 (Thu Nov 14 14:17:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.265 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:17:54,298::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:17:54,307::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:17:54,307::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:17:54,307::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431474, 'last-update-local-ts': 1384431474.307256, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:54,308::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:17:54,308::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431470, 'last-update-local-ts': 1384431474.307256, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:17:54,308::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:18:04,384::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:18:04,645::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:18:04,738::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:18:04,805::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:18:04,806::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:18:04,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:18:04,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:04,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:18:04,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:18:04,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 14:18:04,814::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:04,814::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431484|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431484 (Thu Nov 14 14:18:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:18:04,814::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:18:04,823::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:18:04,823::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:18:04,824::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431484, 'last-update-local-ts': 1384431484.823343, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:04,824::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:18:04,824::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431480, 'last-update-local-ts': 1384431484.823343, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:04,824::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:18:14,887::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:18:15,144::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:18:15,212::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:18:15,271::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:18:15,272::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:18:15,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:18:15,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:15,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:18:15,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:18:15,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 14:18:15,280::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:15,280::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431495|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431495 (Thu Nov 14 14:18:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:18:15,280::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:18:15,291::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:18:15,291::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:18:15,291::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431495, 'last-update-local-ts': 1384431495.2912259, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:15,292::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:18:15,292::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431491, 'last-update-local-ts': 1384431495.2912259, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:15,292::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:18:25,377::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:18:25,626::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:18:25,739::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:18:25,835::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:18:25,835::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:18:25,835::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:18:25,838::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:25,839::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:18:25,839::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:18:25,839::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 14:18:25,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:25,841::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431505|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431505 (Thu Nov 14 14:18:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:18:25,841::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:18:25,855::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:18:25,856::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:18:25,856::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431505, 'last-update-local-ts': 1384431505.8557661, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:25,856::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:18:25,856::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431501, 'last-update-local-ts': 1384431505.8557661, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:25,857::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:18:35,932::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:18:36,187::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:18:36,275::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:18:36,347::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:18:36,347::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:18:36,348::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:18:36,348::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:36,348::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:18:36,349::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:18:36,349::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 14:18:36,349::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:36,350::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431516|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431516 (Thu Nov 14 14:18:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:18:36,350::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:18:36,358::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:18:36,358::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:18:36,359::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431516, 'last-update-local-ts': 1384431516.3585179, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:36,359::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:18:36,359::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431512, 'last-update-local-ts': 1384431516.3585179, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:36,359::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:18:46,441::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:18:46,698::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:18:46,788::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:18:46,849::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:18:46,849::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:18:46,850::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:18:46,850::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:46,850::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:18:46,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:18:46,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.15 >MainThread::INFO::2013-11-14 14:18:46,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:46,852::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431526|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431526 (Thu Nov 14 14:18:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.15 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:18:46,853::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:18:46,866::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:18:46,866::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:18:46,866::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431526, 'last-update-local-ts': 1384431526.8658941, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:46,866::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:18:46,866::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431522, 'last-update-local-ts': 1384431526.8658941, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:46,867::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:18:56,942::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:18:57,189::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:18:57,274::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:18:57,337::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:18:57,338::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:18:57,338::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:18:57,338::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:57,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:18:57,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:18:57,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 14:18:57,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:18:57,340::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431537|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431537 (Thu Nov 14 14:18:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:18:57,340::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:18:57,348::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:18:57,349::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:18:57,349::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431537, 'last-update-local-ts': 1384431537.348556, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:57,349::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:18:57,349::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431533, 'last-update-local-ts': 1384431537.348556, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:18:57,349::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:19:07,422::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:19:07,711::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:19:07,817::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:19:07,878::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:19:07,878::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:19:07,879::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:19:07,879::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:07,879::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7061 >MainThread::INFO::2013-11-14 14:19:07,880::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:19:07,880::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 14:19:07,880::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:07,881::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431547|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431547 (Thu Nov 14 14:19:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7061 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:19:07,881::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:19:07,888::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:19:07,888::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:19:07,889::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431547, 'last-update-local-ts': 1384431547.8883619, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:07,889::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:19:07,889::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431543, 'last-update-local-ts': 1384431547.8883619, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:07,889::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:19:17,965::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:19:18,270::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:19:18,357::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:19:18,419::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:19:18,419::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:19:18,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:19:18,420::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:18,420::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:19:18,420::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:19:18,420::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 14:19:18,421::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:18,421::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431558|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431558 (Thu Nov 14 14:19:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:19:18,421::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:19:18,430::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:19:18,430::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:19:18,430::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431558, 'last-update-local-ts': 1384431558.4299531, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:18,430::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:19:18,430::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431554, 'last-update-local-ts': 1384431558.4299531, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:18,431::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:19:28,508::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:19:28,782::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:19:28,846::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:19:28,906::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:19:28,907::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:19:28,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:19:28,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:28,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:19:28,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:19:28,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 14:19:28,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:28,915::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431568|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431568 (Thu Nov 14 14:19:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:19:28,915::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:19:28,923::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:19:28,924::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:19:28,924::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431568, 'last-update-local-ts': 1384431568.9236889, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:28,924::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:19:28,924::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431564, 'last-update-local-ts': 1384431568.9236889, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:28,925::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:19:38,997::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:19:39,253::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:19:39,331::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:19:39,734::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:19:39,735::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:19:39,735::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:19:39,735::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:39,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:19:39,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:19:39,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 14:19:39,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:39,737::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431579|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431579 (Thu Nov 14 14:19:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:19:39,737::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:19:39,745::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:19:39,745::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:19:39,746::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431579, 'last-update-local-ts': 1384431579.7455189, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:39,746::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:19:39,746::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431574, 'last-update-local-ts': 1384431579.7455189, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:39,746::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:19:49,834::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:19:50,082::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:19:50,169::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:19:50,230::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:19:50,231::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:19:50,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:19:50,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:50,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:19:50,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:19:50,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 14:19:50,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:19:50,234::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431590|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431590 (Thu Nov 14 14:19:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:19:50,234::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:19:50,243::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:19:50,244::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:19:50,244::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431590, 'last-update-local-ts': 1384431590.2436891, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:50,244::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:19:50,244::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431585, 'last-update-local-ts': 1384431590.2436891, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:19:50,245::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:20:00,321::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:20:00,578::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:20:00,668::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:20:00,729::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:20:00,729::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:20:00,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:20:00,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:00,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:20:00,731::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:20:00,731::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 14:20:00,731::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:00,732::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431600|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431600 (Thu Nov 14 14:20:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:20:00,732::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:20:00,740::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:20:00,741::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:20:00,741::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431600, 'last-update-local-ts': 1384431600.7407169, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:00,741::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:20:00,741::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431595, 'last-update-local-ts': 1384431600.7407169, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:00,742::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:20:10,842::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:20:11,103::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:20:11,199::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:20:11,266::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:20:11,267::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:20:11,267::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:20:11,267::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:11,268::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:20:11,268::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:20:11,268::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 14:20:11,269::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:11,270::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431611|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431611 (Thu Nov 14 14:20:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:20:11,270::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:20:11,283::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:20:11,283::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:20:11,283::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431611, 'last-update-local-ts': 1384431611.2830031, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:11,283::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:20:11,284::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431606, 'last-update-local-ts': 1384431611.2830031, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:11,284::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:20:21,357::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:20:21,613::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:20:21,702::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:20:21,762::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:20:21,762::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:20:21,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:20:21,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:21,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:20:21,764::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:20:21,764::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 14:20:21,764::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:21,765::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431621|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431621 (Thu Nov 14 14:20:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:20:21,765::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:20:21,775::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:20:21,776::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:20:21,776::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431621, 'last-update-local-ts': 1384431621.775635, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:21,776::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:20:21,776::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431616, 'last-update-local-ts': 1384431621.775635, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:21,776::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:20:31,849::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:20:32,097::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:20:32,151::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:20:32,210::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:20:32,211::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:20:32,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:20:32,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:32,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:20:32,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:20:32,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 14:20:32,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:32,213::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431632|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431632 (Thu Nov 14 14:20:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:20:32,213::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:20:32,222::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:20:32,222::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:20:32,222::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431632, 'last-update-local-ts': 1384431632.221946, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:32,222::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:20:32,222::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431627, 'last-update-local-ts': 1384431632.221946, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:32,223::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:20:42,311::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:20:42,585::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:20:42,664::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:20:42,726::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:20:42,727::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:20:42,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:20:42,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:42,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:20:42,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:20:42,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 14:20:42,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:42,729::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431642|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431642 (Thu Nov 14 14:20:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:20:42,729::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:20:42,738::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:20:42,738::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:20:42,738::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431642, 'last-update-local-ts': 1384431642.738239, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:42,739::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:20:42,739::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431637, 'last-update-local-ts': 1384431642.738239, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:42,739::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:20:52,826::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:20:53,080::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:20:53,169::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:20:53,229::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:20:53,229::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:20:53,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:20:53,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:53,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:20:53,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:20:53,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 14:20:53,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:20:53,232::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431653|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431653 (Thu Nov 14 14:20:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:20:53,232::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:20:53,240::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:20:53,240::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:20:53,240::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431653, 'last-update-local-ts': 1384431653.2402, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:53,241::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:20:53,241::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431648, 'last-update-local-ts': 1384431653.2402, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:20:53,241::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:21:03,339::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:21:03,591::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:21:03,673::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:21:03,741::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:21:03,741::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:21:03,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:21:03,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:03,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:21:03,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:21:03,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 14:21:03,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:03,744::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431663|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431663 (Thu Nov 14 14:21:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:21:03,744::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:21:03,754::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:21:03,754::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:21:03,754::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431663, 'last-update-local-ts': 1384431663.754266, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:03,755::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:21:03,755::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431658, 'last-update-local-ts': 1384431663.754266, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:03,755::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:21:13,841::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:21:14,100::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:21:14,189::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:21:14,250::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:21:14,250::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:21:14,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:21:14,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:14,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:21:14,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:21:14,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 14:21:14,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:14,258::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431674|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431674 (Thu Nov 14 14:21:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:21:14,259::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:21:14,269::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:21:14,269::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:21:14,269::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431674, 'last-update-local-ts': 1384431674.269191, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:14,270::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:21:14,270::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431669, 'last-update-local-ts': 1384431674.269191, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:14,270::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:21:24,349::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:21:24,598::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:21:24,651::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:21:24,711::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:21:24,711::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:21:24,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:21:24,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:24,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:21:24,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:21:24,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 14:21:24,713::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:24,714::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431684|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431684 (Thu Nov 14 14:21:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:21:24,714::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:21:24,723::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:21:24,723::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:21:24,723::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431684, 'last-update-local-ts': 1384431684.7230611, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:24,723::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:21:24,724::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431679, 'last-update-local-ts': 1384431684.7230611, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:24,724::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:21:34,820::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:21:35,075::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:21:35,163::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:21:35,232::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:21:35,233::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:21:35,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:21:35,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:35,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:21:35,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:21:35,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 14:21:35,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:35,235::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431695|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431695 (Thu Nov 14 14:21:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:21:35,236::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:21:35,247::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:21:35,247::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:21:35,247::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431695, 'last-update-local-ts': 1384431695.2470851, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:35,248::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:21:35,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431690, 'last-update-local-ts': 1384431695.2470851, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:35,248::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:21:45,334::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:21:45,586::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:21:45,678::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:21:45,738::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:21:45,738::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:21:45,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:21:45,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:45,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:21:45,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:21:45,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 14:21:45,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:45,747::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431705|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431705 (Thu Nov 14 14:21:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:21:45,747::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:21:45,757::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:21:45,757::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:21:45,757::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431705, 'last-update-local-ts': 1384431705.757242, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:45,758::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:21:45,758::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431700, 'last-update-local-ts': 1384431705.757242, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:45,758::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:21:55,852::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:21:56,148::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:21:56,231::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:21:56,292::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:21:56,292::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:21:56,293::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:21:56,293::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:56,293::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7066 >MainThread::INFO::2013-11-14 14:21:56,294::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:21:56,294::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 14:21:56,294::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:21:56,295::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431716|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431716 (Thu Nov 14 14:21:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7066 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:21:56,296::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:21:56,304::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:21:56,304::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:21:56,305::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431716, 'last-update-local-ts': 1384431716.3045211, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:56,305::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:21:56,305::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431710, 'last-update-local-ts': 1384431716.3045211, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:21:56,305::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:22:06,389::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:22:06,645::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:22:06,731::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:22:06,792::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:22:06,792::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:22:06,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:22:06,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:06,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:22:06,800::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:22:06,800::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 14:22:06,800::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:06,801::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431726|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431726 (Thu Nov 14 14:22:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:22:06,801::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:22:06,811::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:22:06,811::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:22:06,811::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431726, 'last-update-local-ts': 1384431726.8110659, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:06,811::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:22:06,812::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431721, 'last-update-local-ts': 1384431726.8110659, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:06,812::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:22:16,886::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:22:17,145::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:22:17,208::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:22:17,276::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:22:17,276::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:22:17,276::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:22:17,277::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:17,277::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:22:17,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:22:17,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 14:22:17,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:17,284::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431737|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431737 (Thu Nov 14 14:22:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:22:17,284::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:22:17,307::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:22:17,311::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:22:17,311::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431737, 'last-update-local-ts': 1384431737.307725, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:17,311::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:22:17,312::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431731, 'last-update-local-ts': 1384431737.307725, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:17,312::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:22:27,408::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:22:27,730::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:22:27,948::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:22:28,019::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:22:28,019::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:22:28,020::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:22:28,020::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:28,020::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7054 >MainThread::INFO::2013-11-14 14:22:28,021::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:22:28,021::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 14:22:28,021::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:28,022::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431748|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431748 (Thu Nov 14 14:22:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7054 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:22:28,022::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:22:28,040::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:22:28,041::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:22:28,041::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431748, 'last-update-local-ts': 1384431748.0404971, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:28,041::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:22:28,041::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431742, 'last-update-local-ts': 1384431748.0404971, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:28,041::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:22:38,116::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:22:38,364::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:22:38,432::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:22:38,491::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:22:38,491::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:22:38,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:22:38,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:38,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7066 >MainThread::INFO::2013-11-14 14:22:38,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:22:38,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 14:22:38,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:38,494::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431758|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431758 (Thu Nov 14 14:22:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7066 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:22:38,494::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:22:38,501::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:22:38,501::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:22:38,501::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431758, 'last-update-local-ts': 1384431758.5010409, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:38,501::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:22:38,502::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431752, 'last-update-local-ts': 1384431758.5010409, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:38,502::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:22:48,583::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:22:48,836::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:22:48,920::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:22:48,980::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:22:48,980::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:22:48,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:22:48,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:48,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:22:48,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:22:48,982::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 14:22:48,982::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:48,982::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431768|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431768 (Thu Nov 14 14:22:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:22:48,983::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:22:48,989::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:22:48,990::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:22:48,990::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431768, 'last-update-local-ts': 1384431768.9898479, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:48,990::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:22:48,990::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431763, 'last-update-local-ts': 1384431768.9898479, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:48,991::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:22:59,087::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:22:59,335::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:22:59,412::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:22:59,471::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:22:59,471::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:22:59,472::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:22:59,472::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:59,473::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:22:59,473::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:22:59,473::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 14:22:59,473::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:22:59,474::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431779|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431779 (Thu Nov 14 14:22:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:22:59,475::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:22:59,484::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:22:59,484::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:22:59,484::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431779, 'last-update-local-ts': 1384431779.4840901, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:59,484::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:22:59,485::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431773, 'last-update-local-ts': 1384431779.4840901, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:22:59,485::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:23:09,573::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:23:09,826::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:23:09,913::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:23:09,974::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:23:09,974::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:23:09,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:23:09,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:09,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:23:09,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:23:09,982::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 14:23:09,982::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:09,982::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431789|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431789 (Thu Nov 14 14:23:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:23:09,982::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:23:10,002::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:23:10,002::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:23:10,003::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431789, 'last-update-local-ts': 1384431790.002378, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:10,003::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:23:10,003::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431784, 'last-update-local-ts': 1384431790.002378, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:10,003::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:23:20,075::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:23:20,329::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:23:20,391::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:23:20,450::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:23:20,450::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:23:20,450::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:23:20,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:20,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 14:23:20,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:23:20,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 14:23:20,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:20,453::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431800|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431800 (Thu Nov 14 14:23:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:23:20,453::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:23:20,462::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:23:20,463::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:23:20,463::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431800, 'last-update-local-ts': 1384431800.462677, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:20,463::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:23:20,463::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431794, 'last-update-local-ts': 1384431800.462677, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:20,464::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:23:30,565::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:23:30,816::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:23:30,909::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:23:30,989::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:23:30,990::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:23:30,990::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:23:30,990::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:30,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:23:30,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:23:30,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 14:23:30,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:30,993::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431810|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431810 (Thu Nov 14 14:23:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:23:30,993::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:23:31,008::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:23:31,009::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:23:31,009::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431810, 'last-update-local-ts': 1384431811.0085609, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:31,009::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:23:31,009::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431804, 'last-update-local-ts': 1384431811.0085609, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:31,010::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:23:41,096::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:23:41,342::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:23:41,431::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:23:41,490::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:23:41,491::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:23:41,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:23:41,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:41,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:23:41,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:23:41,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 14:23:41,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:41,493::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431821|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431821 (Thu Nov 14 14:23:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:23:41,493::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:23:41,503::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:23:41,504::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:23:41,504::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431821, 'last-update-local-ts': 1384431821.5036199, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:41,504::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:23:41,504::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431815, 'last-update-local-ts': 1384431821.5036199, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:41,504::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:23:51,590::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:23:51,841::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:23:51,929::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:23:51,989::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:23:51,989::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:23:51,990::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:23:51,990::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:51,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:23:51,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:23:51,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 14:23:51,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:23:51,992::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431831|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431831 (Thu Nov 14 14:23:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:23:51,993::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:23:52,000::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:23:52,000::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:23:52,000::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431831, 'last-update-local-ts': 1384431832.000272, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:52,001::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:23:52,001::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431825, 'last-update-local-ts': 1384431832.000272, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:23:52,001::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:24:02,068::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:24:02,334::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:24:02,428::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:24:02,510::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:24:02,510::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:24:02,511::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:24:02,511::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:02,512::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:24:02,512::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:24:02,513::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.655 >MainThread::INFO::2013-11-14 14:24:02,513::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:02,514::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431842|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431842 (Thu Nov 14 14:24:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.655 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:24:02,514::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:24:02,555::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:24:02,556::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:24:02,556::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431842, 'last-update-local-ts': 1384431842.555764, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:02,556::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:24:02,556::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431836, 'last-update-local-ts': 1384431842.555764, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:02,557::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:24:12,625::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:24:12,952::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:24:13,030::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:24:13,090::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:24:13,090::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:24:13,091::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:24:13,091::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:13,091::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:24:13,092::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:24:13,092::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.55 >MainThread::INFO::2013-11-14 14:24:13,092::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:13,093::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431853|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431853 (Thu Nov 14 14:24:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.55 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:24:13,093::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:24:13,103::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:24:13,103::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:24:13,103::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431853, 'last-update-local-ts': 1384431853.1031089, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:13,103::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:24:13,104::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431846, 'last-update-local-ts': 1384431853.1031089, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:13,104::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:24:23,188::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:24:23,472::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:24:23,528::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:24:23,597::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:24:23,598::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:24:23,598::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:24:23,598::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:23,599::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:24:23,599::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:24:23,599::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.585 >MainThread::INFO::2013-11-14 14:24:23,600::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:23,600::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431863|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431863 (Thu Nov 14 14:24:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.585 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:24:23,600::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:24:23,609::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:24:23,609::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:24:23,609::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431863, 'last-update-local-ts': 1384431863.609174, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:23,610::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:24:23,610::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431857, 'last-update-local-ts': 1384431863.609174, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:23,610::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:24:33,682::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:24:33,932::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:24:34,026::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:24:34,087::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:24:34,087::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:24:34,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:24:34,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:34,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7066 >MainThread::INFO::2013-11-14 14:24:34,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:24:34,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.495 >MainThread::INFO::2013-11-14 14:24:34,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:34,096::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431874|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431874 (Thu Nov 14 14:24:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.495 >engine-health=vm-down >gateway=True >mem-free=7066 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:24:34,096::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:24:34,105::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:24:34,105::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:24:34,106::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431874, 'last-update-local-ts': 1384431874.105469, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:34,106::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:24:34,106::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431867, 'last-update-local-ts': 1384431874.105469, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:34,106::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:24:44,184::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:24:44,439::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:24:44,520::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:24:44,581::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:24:44,581::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:24:44,581::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:24:44,582::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:44,582::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:24:44,582::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:24:44,582::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.42 >MainThread::INFO::2013-11-14 14:24:44,583::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:44,584::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431884|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431884 (Thu Nov 14 14:24:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.42 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:24:44,584::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:24:44,593::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:24:44,593::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:24:44,593::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431884, 'last-update-local-ts': 1384431884.5931749, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:44,594::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:24:44,594::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431878, 'last-update-local-ts': 1384431884.5931749, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:44,594::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:24:54,671::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:24:54,927::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:24:55,014::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:24:55,082::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:24:55,082::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:24:55,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:24:55,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:55,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:24:55,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:24:55,084::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 14:24:55,084::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:24:55,084::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431895|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431895 (Thu Nov 14 14:24:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:24:55,084::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:24:55,093::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:24:55,093::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:24:55,093::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431895, 'last-update-local-ts': 1384431895.093133, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:55,093::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:24:55,094::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431888, 'last-update-local-ts': 1384431895.093133, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:24:55,094::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:25:05,182::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:25:05,447::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:25:05,521::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:25:05,580::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:25:05,581::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:25:05,587::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:25:05,588::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:05,588::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:25:05,588::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:25:05,588::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 14:25:05,589::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:05,589::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431905|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431905 (Thu Nov 14 14:25:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:25:05,590::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:25:05,600::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:25:05,600::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:25:05,600::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431905, 'last-update-local-ts': 1384431905.60006, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:05,600::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:25:05,601::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431899, 'last-update-local-ts': 1384431905.60006, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:05,601::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:25:15,674::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:25:15,928::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:25:16,025::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:25:16,085::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:25:16,086::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:25:16,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:25:16,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:16,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:25:16,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:25:16,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 14:25:16,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:16,088::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431916|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431916 (Thu Nov 14 14:25:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:25:16,088::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:25:16,097::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:25:16,097::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:25:16,097::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431916, 'last-update-local-ts': 1384431916.0970299, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:16,097::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:25:16,098::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431909, 'last-update-local-ts': 1384431916.0970299, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:16,098::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:25:26,174::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:25:26,433::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:25:26,532::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:25:26,592::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:25:26,592::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:25:26,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:25:26,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:26,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:25:26,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:25:26,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 14:25:26,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:26,595::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431926|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431926 (Thu Nov 14 14:25:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:25:26,595::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:25:26,604::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:25:26,605::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:25:26,605::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431926, 'last-update-local-ts': 1384431926.6046171, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:26,605::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:25:26,605::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431920, 'last-update-local-ts': 1384431926.6046171, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:26,605::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:25:36,690::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:25:36,944::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:25:37,018::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:25:37,085::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:25:37,085::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:25:37,085::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:25:37,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:37,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:25:37,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:25:37,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 14:25:37,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:37,087::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431937|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431937 (Thu Nov 14 14:25:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:25:37,087::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:25:37,095::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:25:37,095::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:25:37,095::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431937, 'last-update-local-ts': 1384431937.095067, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:37,095::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:25:37,096::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431930, 'last-update-local-ts': 1384431937.095067, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:37,096::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:25:47,180::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:25:47,432::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:25:47,509::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:25:47,569::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:25:47,569::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:25:47,576::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:25:47,577::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:47,577::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:25:47,577::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:25:47,577::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 14:25:47,578::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:47,578::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431947|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431947 (Thu Nov 14 14:25:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:25:47,578::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:25:47,589::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:25:47,589::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:25:47,589::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431947, 'last-update-local-ts': 1384431947.5891509, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:47,590::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:25:47,590::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431940, 'last-update-local-ts': 1384431947.5891509, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:47,590::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:25:57,682::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:25:57,975::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:25:58,053::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:25:58,114::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:25:58,114::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:25:58,114::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:25:58,115::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:58,115::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7057 >MainThread::INFO::2013-11-14 14:25:58,115::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:25:58,116::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 14:25:58,116::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:25:58,117::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431958|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431958 (Thu Nov 14 14:25:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7057 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:25:58,117::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:25:58,124::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:25:58,124::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:25:58,124::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431958, 'last-update-local-ts': 1384431958.124228, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:58,125::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:25:58,125::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431951, 'last-update-local-ts': 1384431958.124228, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:25:58,125::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:26:08,225::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:26:08,481::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:26:08,561::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:26:08,621::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:26:08,621::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:26:08,622::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:26:08,622::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:08,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:26:08,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:26:08,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 14:26:08,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:08,624::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431968|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431968 (Thu Nov 14 14:26:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:26:08,624::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:26:08,630::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:26:08,631::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:26:08,631::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431968, 'last-update-local-ts': 1384431968.630877, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:08,631::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:26:08,631::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431961, 'last-update-local-ts': 1384431968.630877, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:08,632::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:26:18,719::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:26:18,988::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:26:19,078::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:26:19,140::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:26:19,140::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:26:19,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:26:19,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:19,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:26:19,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:26:19,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 14:26:19,142::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:19,143::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431979|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431979 (Thu Nov 14 14:26:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:26:19,143::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:26:19,151::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:26:19,152::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:26:19,152::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431979, 'last-update-local-ts': 1384431979.1516731, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:19,152::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:26:19,152::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431972, 'last-update-local-ts': 1384431979.1516731, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:19,153::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:26:29,250::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:26:29,502::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:26:29,583::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:26:29,649::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:26:29,650::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:26:29,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:26:29,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:29,651::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:26:29,651::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:26:29,651::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 14:26:29,651::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:29,652::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384431989|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384431989 (Thu Nov 14 14:26:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:26:29,652::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:26:29,661::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:26:29,661::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:26:29,661::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384431989, 'last-update-local-ts': 1384431989.6609659, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:29,661::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:26:29,661::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431982, 'last-update-local-ts': 1384431989.6609659, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:29,662::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:26:39,735::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:26:39,996::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:26:40,055::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:26:40,122::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:26:40,122::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:26:40,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:26:40,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:40,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:26:40,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:26:40,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 14:26:40,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:40,124::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432000|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432000 (Thu Nov 14 14:26:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:26:40,125::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:26:40,133::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:26:40,133::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:26:40,134::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432000, 'last-update-local-ts': 1384432000.133456, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:40,134::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:26:40,134::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384431993, 'last-update-local-ts': 1384432000.133456, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:40,134::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:26:50,216::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:26:50,469::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:26:50,558::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:26:50,618::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:26:50,618::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:26:50,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:26:50,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:50,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:26:50,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:26:50,620::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 14:26:50,620::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:26:50,621::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432010|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432010 (Thu Nov 14 14:26:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:26:50,621::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:26:50,630::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:26:50,630::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:26:50,630::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432010, 'last-update-local-ts': 1384432010.630198, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:50,631::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:26:50,631::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432003, 'last-update-local-ts': 1384432010.630198, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:26:50,631::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:27:00,723::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:27:00,973::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:27:01,055::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:27:01,122::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:27:01,122::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:27:01,122::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:27:01,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:01,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:27:01,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:27:01,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 14:27:01,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:01,124::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432021|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432021 (Thu Nov 14 14:27:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:27:01,124::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:27:01,133::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:27:01,133::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:27:01,134::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432021, 'last-update-local-ts': 1384432021.133393, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:01,134::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:27:01,134::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432014, 'last-update-local-ts': 1384432021.133393, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:01,134::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:27:11,221::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:27:11,480::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:27:11,577::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:27:11,638::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:27:11,638::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:27:11,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:27:11,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:11,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:27:11,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:27:11,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 14:27:11,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:11,641::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432031|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432031 (Thu Nov 14 14:27:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:27:11,641::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:27:11,650::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:27:11,650::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:27:11,650::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432031, 'last-update-local-ts': 1384432031.6499219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:11,650::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:27:11,650::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432024, 'last-update-local-ts': 1384432031.6499219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:11,651::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:27:21,735::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:27:21,982::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:27:22,039::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:27:22,098::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:27:22,099::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:27:22,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:27:22,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:22,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:27:22,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:27:22,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 14:27:22,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:22,101::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432042|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432042 (Thu Nov 14 14:27:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:27:22,102::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:27:22,110::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:27:22,111::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:27:22,111::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432042, 'last-update-local-ts': 1384432042.1106601, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:22,111::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:27:22,111::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432035, 'last-update-local-ts': 1384432042.1106601, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:22,112::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:27:32,207::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:27:32,509::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:27:32,637::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:27:32,705::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:27:32,706::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:27:32,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:27:32,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:32,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:27:32,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:27:32,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 14:27:32,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:32,713::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432052|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432052 (Thu Nov 14 14:27:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:27:32,713::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:27:32,738::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:27:32,739::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:27:32,739::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432052, 'last-update-local-ts': 1384432052.7388079, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:32,739::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:27:32,739::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432045, 'last-update-local-ts': 1384432052.7388079, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:32,740::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:27:42,820::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:27:43,068::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:27:43,144::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:27:43,204::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:27:43,204::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:27:43,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:27:43,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:43,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:27:43,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:27:43,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 14:27:43,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:43,213::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432063|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432063 (Thu Nov 14 14:27:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:27:43,213::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:27:43,222::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:27:43,222::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:27:43,222::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432063, 'last-update-local-ts': 1384432063.222079, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:43,222::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:27:43,223::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432055, 'last-update-local-ts': 1384432063.222079, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:43,223::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:27:53,297::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:27:53,548::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:27:53,628::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:27:53,688::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:27:53,688::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:27:53,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:27:53,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:53,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:27:53,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:27:53,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 14:27:53,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:27:53,691::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432073|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432073 (Thu Nov 14 14:27:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:27:53,691::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:27:53,700::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:27:53,700::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:27:53,700::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432073, 'last-update-local-ts': 1384432073.7001641, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:53,701::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:27:53,701::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432066, 'last-update-local-ts': 1384432073.7001641, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:27:53,701::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:28:03,805::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:28:04,057::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:28:04,140::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:28:04,208::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:28:04,208::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:28:04,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:28:04,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:04,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:28:04,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:28:04,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 14:28:04,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:04,210::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432084|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432084 (Thu Nov 14 14:28:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:28:04,211::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:28:04,219::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:28:04,220::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:28:04,220::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432084, 'last-update-local-ts': 1384432084.2198851, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:04,220::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:28:04,220::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432076, 'last-update-local-ts': 1384432084.2198851, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:04,221::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:28:14,307::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:28:14,568::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:28:14,649::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:28:14,710::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:28:14,710::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:28:14,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:28:14,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:14,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:28:14,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:28:14,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 14:28:14,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:14,718::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432094|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432094 (Thu Nov 14 14:28:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:28:14,718::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:28:14,728::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:28:14,729::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:28:14,729::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432094, 'last-update-local-ts': 1384432094.728756, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:14,729::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:28:14,729::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432087, 'last-update-local-ts': 1384432094.728756, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:14,730::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:28:24,804::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:28:25,052::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:28:25,105::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:28:25,172::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:28:25,172::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:28:25,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:28:25,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:25,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:28:25,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:28:25,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 14:28:25,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:25,179::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432105|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432105 (Thu Nov 14 14:28:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:28:25,180::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:28:25,192::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:28:25,192::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:28:25,193::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432105, 'last-update-local-ts': 1384432105.1924019, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:25,193::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:28:25,193::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432097, 'last-update-local-ts': 1384432105.1924019, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:25,193::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:28:35,302::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:28:35,554::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:28:35,637::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:28:35,710::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:28:35,710::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:28:35,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:28:35,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:35,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:28:35,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:28:35,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 14:28:35,713::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:35,713::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432115|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432115 (Thu Nov 14 14:28:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:28:35,713::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:28:35,724::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:28:35,724::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:28:35,724::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432115, 'last-update-local-ts': 1384432115.7238979, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:35,724::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:28:35,725::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432108, 'last-update-local-ts': 1384432115.7238979, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:35,725::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:28:45,811::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:28:46,066::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:28:46,150::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:28:46,210::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:28:46,210::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:28:46,217::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:28:46,217::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:46,217::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:28:46,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:28:46,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.01 >MainThread::INFO::2013-11-14 14:28:46,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:46,219::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432126|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432126 (Thu Nov 14 14:28:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.01 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:28:46,219::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:28:46,228::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:28:46,229::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:28:46,229::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432126, 'last-update-local-ts': 1384432126.228662, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:46,229::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:28:46,229::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432118, 'last-update-local-ts': 1384432126.228662, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:46,230::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:28:56,316::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:28:56,565::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:28:56,665::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:28:56,724::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:28:56,725::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:28:56,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:28:56,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:56,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:28:56,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:28:56,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.01 >MainThread::INFO::2013-11-14 14:28:56,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:28:56,728::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432136|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432136 (Thu Nov 14 14:28:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.01 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:28:56,728::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:28:56,736::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:28:56,737::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:28:56,737::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432136, 'last-update-local-ts': 1384432136.736886, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:56,737::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:28:56,737::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432129, 'last-update-local-ts': 1384432136.736886, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:28:56,738::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:29:06,805::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:29:07,067::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:29:07,166::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:29:07,227::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:29:07,227::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:29:07,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:29:07,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:07,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:29:07,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:29:07,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.01 >MainThread::INFO::2013-11-14 14:29:07,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:07,230::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432147|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432147 (Thu Nov 14 14:29:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.01 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:29:07,230::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:29:07,239::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:29:07,239::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:29:07,239::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432147, 'last-update-local-ts': 1384432147.2390571, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:07,239::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:29:07,240::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432139, 'last-update-local-ts': 1384432147.2390571, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:07,240::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:29:17,307::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:29:17,562::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:29:17,623::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:29:17,681::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:29:17,681::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:29:17,682::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:29:17,682::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:17,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:29:17,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:29:17,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.005 >MainThread::INFO::2013-11-14 14:29:17,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:17,684::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432157|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432157 (Thu Nov 14 14:29:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.005 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:29:17,684::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:29:17,691::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:29:17,692::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:29:17,692::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432157, 'last-update-local-ts': 1384432157.6917629, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:17,692::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:29:17,692::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432149, 'last-update-local-ts': 1384432157.6917629, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:17,693::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:29:27,788::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:29:28,039::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:29:28,139::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:29:28,209::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:29:28,209::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:29:28,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:29:28,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:28,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 14:29:28,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:29:28,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 14:29:28,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:28,212::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432168|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432168 (Thu Nov 14 14:29:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:29:28,212::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:29:28,219::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:29:28,220::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:29:28,220::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432168, 'last-update-local-ts': 1384432168.2196779, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:28,220::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:29:28,220::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432160, 'last-update-local-ts': 1384432168.2196779, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:28,221::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:29:38,303::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:29:38,558::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:29:38,657::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:29:38,717::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:29:38,717::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:29:38,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:29:38,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:38,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:29:38,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:29:38,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 14:29:38,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:38,726::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432178|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432178 (Thu Nov 14 14:29:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:29:38,726::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:29:38,734::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:29:38,734::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:29:38,735::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432178, 'last-update-local-ts': 1384432178.7343769, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:38,735::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:29:38,735::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432170, 'last-update-local-ts': 1384432178.7343769, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:38,735::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:29:48,821::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:29:49,074::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:29:49,167::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:29:49,229::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:29:49,229::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:29:49,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:29:49,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:49,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:29:49,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:29:49,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 14:29:49,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:49,232::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432189|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432189 (Thu Nov 14 14:29:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:29:49,232::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:29:49,239::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:29:49,239::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:29:49,239::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432189, 'last-update-local-ts': 1384432189.239186, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:49,240::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:29:49,240::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432181, 'last-update-local-ts': 1384432189.239186, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:49,240::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:29:59,314::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:29:59,563::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:29:59,639::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:29:59,698::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:29:59,698::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:29:59,699::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:29:59,699::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:59,699::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:29:59,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:29:59,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 14:29:59,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:29:59,701::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432199|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432199 (Thu Nov 14 14:29:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:29:59,701::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:29:59,709::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:29:59,710::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:29:59,710::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432199, 'last-update-local-ts': 1384432199.709718, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:59,710::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:29:59,710::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432191, 'last-update-local-ts': 1384432199.709718, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:29:59,711::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:30:09,802::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:30:10,056::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:30:10,179::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:30:10,242::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:30:10,242::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:30:10,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:30:10,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:10,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:30:10,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:30:10,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 14:30:10,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:10,245::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432210|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432210 (Thu Nov 14 14:30:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:30:10,245::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:30:10,254::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:30:10,255::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:30:10,255::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432210, 'last-update-local-ts': 1384432210.254844, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:10,255::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:30:10,255::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432212, 'last-update-local-ts': 1384432210.254844, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:10,256::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:30:20,337::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:30:20,587::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:30:20,684::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:30:20,744::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:30:20,745::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:30:20,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:30:20,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:20,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:30:20,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:30:20,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 14:30:20,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:20,747::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432220|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432220 (Thu Nov 14 14:30:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:30:20,747::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:30:20,756::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:30:20,756::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:30:20,756::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432220, 'last-update-local-ts': 1384432220.7560489, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:20,756::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:30:20,757::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432223, 'last-update-local-ts': 1384432220.7560489, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:20,757::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:30:30,824::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:30:31,081::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:30:31,176::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:30:31,237::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:30:31,237::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:30:31,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:30:31,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:31,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:30:31,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:30:31,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 14:30:31,239::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:31,240::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432231|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432231 (Thu Nov 14 14:30:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:30:31,240::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:30:31,249::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:30:31,249::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:30:31,249::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432231, 'last-update-local-ts': 1384432231.2491381, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:31,250::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:30:31,250::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432233, 'last-update-local-ts': 1384432231.2491381, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:31,250::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:30:41,328::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:30:41,590::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:30:41,682::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:30:41,745::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:30:41,745::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:30:41,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:30:41,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:41,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 14:30:41,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:30:41,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 14:30:41,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:41,747::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432241|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432241 (Thu Nov 14 14:30:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:30:41,747::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:30:41,754::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:30:41,754::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:30:41,755::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432241, 'last-update-local-ts': 1384432241.7544391, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:41,755::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:30:41,755::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432244, 'last-update-local-ts': 1384432241.7544391, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:41,755::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:30:51,820::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:30:52,078::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:30:52,146::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:30:52,205::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:30:52,205::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:30:52,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:30:52,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:52,213::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:30:52,213::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:30:52,213::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 14:30:52,213::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:30:52,214::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432252|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432252 (Thu Nov 14 14:30:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:30:52,214::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:30:52,224::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:30:52,225::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:30:52,225::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432252, 'last-update-local-ts': 1384432252.2247469, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:52,225::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:30:52,225::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432254, 'last-update-local-ts': 1384432252.2247469, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:30:52,226::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:31:02,311::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:31:02,563::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:31:02,660::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:31:02,727::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:31:02,727::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:31:02,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:31:02,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:02,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:31:02,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:31:02,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 14:31:02,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:02,731::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432262|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432262 (Thu Nov 14 14:31:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:31:02,731::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:31:02,739::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:31:02,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:31:02,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432262, 'last-update-local-ts': 1384432262.7397611, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:02,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:31:02,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432264, 'last-update-local-ts': 1384432262.7397611, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:02,741::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:31:12,829::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:31:13,089::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:31:13,185::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:31:13,246::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:31:13,246::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:31:13,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:31:13,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:13,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:31:13,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:31:13,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 14:31:13,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:13,255::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432273|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432273 (Thu Nov 14 14:31:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:31:13,255::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:31:13,265::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:31:13,265::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:31:13,265::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432273, 'last-update-local-ts': 1384432273.265094, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:13,265::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:31:13,266::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432275, 'last-update-local-ts': 1384432273.265094, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:13,266::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:31:23,343::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:31:23,596::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:31:23,677::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:31:23,737::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:31:23,738::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:31:23,738::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:31:23,738::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:23,739::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7062 >MainThread::INFO::2013-11-14 14:31:23,739::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:31:23,739::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 14:31:23,739::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:23,741::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432283|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432283 (Thu Nov 14 14:31:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7062 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:31:23,741::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:31:23,749::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:31:23,749::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:31:23,749::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432283, 'last-update-local-ts': 1384432283.749284, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:23,750::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:31:23,750::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432285, 'last-update-local-ts': 1384432283.749284, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:23,750::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:31:33,819::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:31:34,116::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:31:34,252::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:31:34,314::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:31:34,315::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:31:34,315::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:31:34,315::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:34,316::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:31:34,316::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:31:34,316::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 14:31:34,316::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:34,317::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432294|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432294 (Thu Nov 14 14:31:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:31:34,317::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:31:34,326::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:31:34,327::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:31:34,327::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432294, 'last-update-local-ts': 1384432294.326782, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:34,327::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:31:34,327::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432296, 'last-update-local-ts': 1384432294.326782, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:34,328::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:31:44,418::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:31:44,692::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:31:44,756::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:31:44,814::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:31:44,815::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:31:44,821::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:31:44,821::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:44,822::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:31:44,822::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:31:44,822::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 14:31:44,823::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:44,823::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432304|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432304 (Thu Nov 14 14:31:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:31:44,823::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:31:44,834::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:31:44,835::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:31:44,835::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432304, 'last-update-local-ts': 1384432304.8345361, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:44,835::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:31:44,835::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432306, 'last-update-local-ts': 1384432304.8345361, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:44,835::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:31:54,908::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:31:55,157::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:31:55,222::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:31:55,282::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:31:55,282::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:31:55,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:31:55,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:55,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:31:55,284::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:31:55,284::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 14:31:55,284::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:31:55,285::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432315|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432315 (Thu Nov 14 14:31:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:31:55,285::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:31:55,294::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:31:55,294::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:31:55,294::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432315, 'last-update-local-ts': 1384432315.2941239, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:55,294::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:31:55,295::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432317, 'last-update-local-ts': 1384432315.2941239, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:31:55,295::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:32:05,389::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:32:05,648::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:32:05,740::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:32:05,801::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:32:05,802::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:32:05,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:32:05,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:05,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:32:05,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:32:05,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 14:32:05,804::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:05,804::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432325|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432325 (Thu Nov 14 14:32:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:32:05,804::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:32:05,815::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:32:05,816::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:32:05,816::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432325, 'last-update-local-ts': 1384432325.815871, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:05,816::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:32:05,816::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432327, 'last-update-local-ts': 1384432325.815871, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:05,817::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:32:15,903::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:32:16,155::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:32:16,234::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:32:16,294::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:32:16,295::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:32:16,301::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:32:16,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:16,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:32:16,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:32:16,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.15 >MainThread::INFO::2013-11-14 14:32:16,303::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:16,303::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432336|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432336 (Thu Nov 14 14:32:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.15 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:32:16,303::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:32:16,312::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:32:16,313::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:32:16,313::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432336, 'last-update-local-ts': 1384432336.3128691, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:16,313::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:32:16,313::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432338, 'last-update-local-ts': 1384432336.3128691, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:16,314::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:32:26,394::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:32:26,649::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:32:26,734::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:32:26,794::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:32:26,795::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:32:26,795::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:32:26,795::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:26,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:32:26,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:32:26,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 14:32:26,797::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:26,798::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432346|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432346 (Thu Nov 14 14:32:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:32:26,798::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:32:26,806::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:32:26,806::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:32:26,806::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432346, 'last-update-local-ts': 1384432346.8060889, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:26,806::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:32:26,807::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432348, 'last-update-local-ts': 1384432346.8060889, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:26,807::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:32:36,880::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:32:37,133::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:32:37,191::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:32:37,252::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:32:37,253::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:32:37,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:32:37,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:37,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:32:37,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:32:37,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 14:32:37,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:37,255::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432357|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432357 (Thu Nov 14 14:32:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:32:37,255::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:32:37,264::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:32:37,264::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:32:37,264::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432357, 'last-update-local-ts': 1384432357.2638941, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:37,264::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:32:37,264::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432359, 'last-update-local-ts': 1384432357.2638941, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:37,265::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:32:47,350::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:32:47,599::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:32:47,682::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:32:47,740::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:32:47,741::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:32:47,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:32:47,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:47,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:32:47,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:32:47,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 14:32:47,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:47,749::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432367|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432367 (Thu Nov 14 14:32:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:32:47,749::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:32:47,760::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:32:47,761::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:32:47,761::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432367, 'last-update-local-ts': 1384432367.760879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:47,761::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:32:47,761::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432369, 'last-update-local-ts': 1384432367.760879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:47,762::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:32:57,848::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:32:58,117::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:32:58,217::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:32:58,278::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:32:58,278::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:32:58,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:32:58,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:58,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 14:32:58,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:32:58,280::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 14:32:58,280::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:32:58,281::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432378|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432378 (Thu Nov 14 14:32:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:32:58,281::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:32:58,288::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:32:58,288::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:32:58,289::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432378, 'last-update-local-ts': 1384432378.2884221, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:58,289::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:32:58,289::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432379, 'last-update-local-ts': 1384432378.2884221, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:32:58,289::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:33:08,378::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:33:08,642::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:33:08,735::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:33:08,807::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:33:08,807::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:33:08,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:33:08,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:08,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:33:08,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:33:08,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 14:33:08,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:08,809::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432388|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432388 (Thu Nov 14 14:33:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:33:08,810::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:33:08,817::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:33:08,817::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:33:08,818::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432388, 'last-update-local-ts': 1384432388.817488, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:08,818::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:33:08,818::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432390, 'last-update-local-ts': 1384432388.817488, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:08,818::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:33:18,889::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:33:19,150::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:33:19,237::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:33:19,298::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:33:19,298::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:33:19,299::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:33:19,299::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:19,299::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:33:19,300::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:33:19,300::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 14:33:19,300::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:19,301::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432399|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432399 (Thu Nov 14 14:33:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:33:19,301::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:33:19,312::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:33:19,312::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:33:19,312::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432399, 'last-update-local-ts': 1384432399.312016, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:19,312::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:33:19,313::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432400, 'last-update-local-ts': 1384432399.312016, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:19,313::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:33:29,390::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:33:29,650::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:33:29,711::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:33:29,772::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:33:29,772::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:33:29,772::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:33:29,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:29,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:33:29,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:33:29,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 14:33:29,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:29,774::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432409|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432409 (Thu Nov 14 14:33:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:33:29,774::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:33:29,783::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:33:29,783::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:33:29,783::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432409, 'last-update-local-ts': 1384432409.783329, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:29,784::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:33:29,784::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432411, 'last-update-local-ts': 1384432409.783329, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:29,784::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:33:39,859::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:33:40,113::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:33:40,195::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:33:40,255::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:33:40,255::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:33:40,256::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:33:40,256::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:40,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:33:40,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:33:40,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 14:33:40,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:40,258::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432420|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432420 (Thu Nov 14 14:33:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:33:40,258::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:33:40,267::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:33:40,268::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:33:40,268::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432420, 'last-update-local-ts': 1384432420.267633, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:40,268::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:33:40,268::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432421, 'last-update-local-ts': 1384432420.267633, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:40,268::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:33:50,351::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:33:50,606::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:33:50,700::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:33:50,761::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:33:50,762::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:33:50,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:33:50,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:50,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:33:50,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:33:50,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 14:33:50,764::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:33:50,765::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432430|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432430 (Thu Nov 14 14:33:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:33:50,765::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:33:50,774::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:33:50,774::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:33:50,774::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432430, 'last-update-local-ts': 1384432430.7743001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:50,775::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:33:50,775::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432432, 'last-update-local-ts': 1384432430.7743001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:33:50,775::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:34:00,849::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:34:01,103::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:34:01,182::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:34:01,242::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:34:01,243::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:34:01,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:34:01,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:01,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:34:01,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:34:01,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 14:34:01,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:01,245::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432441|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432441 (Thu Nov 14 14:34:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:34:01,245::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:34:01,253::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:34:01,253::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:34:01,254::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432441, 'last-update-local-ts': 1384432441.2533751, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:01,254::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:34:01,254::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432442, 'last-update-local-ts': 1384432441.2533751, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:01,254::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:34:11,328::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:34:11,585::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:34:11,675::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:34:11,741::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:34:11,741::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:34:11,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:34:11,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:11,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:34:11,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:34:11,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.34 >MainThread::INFO::2013-11-14 14:34:11,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:11,744::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432451|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432451 (Thu Nov 14 14:34:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.34 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:34:11,745::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:34:11,754::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:34:11,754::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:34:11,755::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432451, 'last-update-local-ts': 1384432451.754524, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:11,755::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:34:11,755::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432453, 'last-update-local-ts': 1384432451.754524, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:11,755::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:34:21,831::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:34:22,084::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:34:22,144::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:34:22,206::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:34:22,206::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:34:22,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:34:22,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:22,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:34:22,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:34:22,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.325 >MainThread::INFO::2013-11-14 14:34:22,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:22,209::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432462|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432462 (Thu Nov 14 14:34:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.325 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:34:22,209::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:34:22,217::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:34:22,218::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:34:22,218::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432462, 'last-update-local-ts': 1384432462.2178071, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:22,218::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:34:22,218::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432463, 'last-update-local-ts': 1384432462.2178071, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:22,219::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:34:32,296::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:34:32,547::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:34:32,635::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:34:32,694::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:34:32,695::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:34:32,695::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:34:32,695::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:32,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:34:32,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:34:32,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.275 >MainThread::INFO::2013-11-14 14:34:32,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:32,697::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432472|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432472 (Thu Nov 14 14:34:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.275 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:34:32,697::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:34:32,706::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:34:32,706::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:34:32,706::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432472, 'last-update-local-ts': 1384432472.706013, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:32,706::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:34:32,707::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432474, 'last-update-local-ts': 1384432472.706013, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:32,707::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:34:42,784::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:34:43,052::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:34:43,148::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:34:43,208::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:34:43,208::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:34:43,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:34:43,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:43,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:34:43,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:34:43,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 14:34:43,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:43,211::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432483|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432483 (Thu Nov 14 14:34:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:34:43,211::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:34:43,220::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:34:43,220::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:34:43,221::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432483, 'last-update-local-ts': 1384432483.220439, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:43,221::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:34:43,221::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432484, 'last-update-local-ts': 1384432483.220439, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:43,221::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:34:53,316::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:34:53,574::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:34:53,648::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:34:53,708::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:34:53,708::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:34:53,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:34:53,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:53,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:34:53,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:34:53,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 14:34:53,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:34:53,710::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432493|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432493 (Thu Nov 14 14:34:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:34:53,711::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:34:53,719::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:34:53,720::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:34:53,720::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432493, 'last-update-local-ts': 1384432493.7195749, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:53,720::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:34:53,720::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432494, 'last-update-local-ts': 1384432493.7195749, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:34:53,720::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:35:03,807::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:35:04,067::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:35:04,153::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:35:04,213::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:35:04,214::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:35:04,220::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:35:04,220::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:04,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:35:04,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:35:04,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 14:35:04,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:04,222::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432504|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432504 (Thu Nov 14 14:35:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:35:04,222::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:35:04,231::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:35:04,232::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:35:04,232::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432504, 'last-update-local-ts': 1384432504.231818, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:04,232::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:35:04,232::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432505, 'last-update-local-ts': 1384432504.231818, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:04,233::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:35:14,315::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:35:14,579::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:35:14,639::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:35:14,699::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:35:14,699::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:35:14,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:35:14,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:14,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:35:14,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:35:14,701::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 14:35:14,701::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:14,703::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432514|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432514 (Thu Nov 14 14:35:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:35:14,703::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:35:14,712::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:35:14,712::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:35:14,712::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432514, 'last-update-local-ts': 1384432514.7119291, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:14,712::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:35:14,712::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432515, 'last-update-local-ts': 1384432514.7119291, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:14,713::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:35:24,804::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:35:25,052::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:35:25,131::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:35:25,198::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:35:25,198::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:35:25,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:35:25,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:25,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:35:25,200::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:35:25,200::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 14:35:25,200::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:25,201::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432525|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432525 (Thu Nov 14 14:35:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:35:25,201::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:35:25,209::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:35:25,210::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:35:25,210::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432525, 'last-update-local-ts': 1384432525.209563, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:25,210::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:35:25,210::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432526, 'last-update-local-ts': 1384432525.209563, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:25,210::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:35:35,292::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:35:35,549::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:35:35,641::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:35:35,704::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:35:35,704::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:35:35,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:35:35,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:35,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:35:35,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:35:35,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 14:35:35,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:35,713::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432535|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432535 (Thu Nov 14 14:35:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:35:35,713::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:35:35,723::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:35:35,723::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:35:35,723::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432535, 'last-update-local-ts': 1384432535.7231901, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:35,724::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:35:35,724::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432536, 'last-update-local-ts': 1384432535.7231901, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:35,724::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:35:45,816::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:35:46,064::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:35:46,137::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:35:46,198::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:35:46,199::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:35:46,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:35:46,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:46,200::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:35:46,200::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:35:46,200::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 14:35:46,201::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:46,202::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432546|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432546 (Thu Nov 14 14:35:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:35:46,202::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:35:46,210::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:35:46,211::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:35:46,211::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432546, 'last-update-local-ts': 1384432546.210876, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:46,211::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:35:46,211::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432547, 'last-update-local-ts': 1384432546.210876, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:46,212::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:35:56,291::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:35:56,548::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:35:56,629::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:35:56,690::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:35:56,690::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:35:56,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:35:56,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:56,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:35:56,698::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:35:56,698::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.22 >MainThread::INFO::2013-11-14 14:35:56,698::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:35:56,699::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432556|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432556 (Thu Nov 14 14:35:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.22 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:35:56,699::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:35:56,707::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:35:56,708::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:35:56,708::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432556, 'last-update-local-ts': 1384432556.7075541, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:56,708::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:35:56,708::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432557, 'last-update-local-ts': 1384432556.7075541, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:35:56,708::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:36:06,782::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:36:07,037::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:36:07,088::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:36:07,146::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:36:07,147::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:36:07,147::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:36:07,147::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:07,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:36:07,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:36:07,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.22 >MainThread::INFO::2013-11-14 14:36:07,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:07,150::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432567|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432567 (Thu Nov 14 14:36:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.22 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:36:07,150::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:36:07,159::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:36:07,159::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:36:07,159::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432567, 'last-update-local-ts': 1384432567.1590209, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:07,159::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:36:07,160::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432557, 'last-update-local-ts': 1384432556.7075541, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:07,160::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:36:17,251::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:36:17,503::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:36:17,586::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:36:17,654::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:36:17,654::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:36:17,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:36:17,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:17,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:36:17,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:36:17,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.37 >MainThread::INFO::2013-11-14 14:36:17,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:17,657::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432577|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432577 (Thu Nov 14 14:36:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.37 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:36:17,657::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:36:17,665::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:36:17,665::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:36:17,666::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432577, 'last-update-local-ts': 1384432577.665518, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:17,666::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:36:17,666::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432572, 'last-update-local-ts': 1384432577.665518, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:17,666::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:36:27,752::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:36:28,011::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:36:28,093::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:36:28,152::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:36:28,153::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:36:28,159::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:36:28,159::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:28,160::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:36:28,160::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:36:28,160::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.315 >MainThread::INFO::2013-11-14 14:36:28,161::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:28,161::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432588|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432588 (Thu Nov 14 14:36:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.315 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:36:28,161::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:36:28,171::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:36:28,171::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:36:28,171::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432588, 'last-update-local-ts': 1384432588.1712601, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:28,172::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:36:28,172::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432583, 'last-update-local-ts': 1384432588.1712601, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:28,172::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:36:38,258::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:36:38,577::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:36:38,674::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:36:38,736::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:36:38,736::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:36:38,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:36:38,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:38,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7058 >MainThread::INFO::2013-11-14 14:36:38,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:36:38,738::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.34 >MainThread::INFO::2013-11-14 14:36:38,738::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:38,739::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432598|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432598 (Thu Nov 14 14:36:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.34 >engine-health=vm-down >gateway=True >mem-free=7058 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:36:38,739::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:36:38,746::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:36:38,746::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:36:38,747::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432598, 'last-update-local-ts': 1384432598.7465351, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:38,747::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:36:38,747::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432593, 'last-update-local-ts': 1384432598.7465351, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:38,747::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:36:48,838::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:36:49,095::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:36:49,170::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:36:49,238::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:36:49,238::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:36:49,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:36:49,239::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:49,239::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:36:49,239::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:36:49,240::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 14:36:49,240::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:49,240::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432609|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432609 (Thu Nov 14 14:36:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:36:49,240::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:36:49,247::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:36:49,247::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:36:49,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432609, 'last-update-local-ts': 1384432609.247371, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:49,248::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:36:49,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432604, 'last-update-local-ts': 1384432609.247371, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:49,248::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:36:59,326::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:36:59,581::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:36:59,635::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:36:59,695::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:36:59,695::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:36:59,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:36:59,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:59,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:36:59,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:36:59,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.245 >MainThread::INFO::2013-11-14 14:36:59,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:36:59,698::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432619|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432619 (Thu Nov 14 14:36:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.245 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:36:59,698::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:36:59,705::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:36:59,706::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:36:59,706::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432619, 'last-update-local-ts': 1384432619.705672, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:59,706::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:36:59,706::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432614, 'last-update-local-ts': 1384432619.705672, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:36:59,707::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:37:09,806::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:37:10,054::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:37:10,143::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:37:10,205::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:37:10,206::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:37:10,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:37:10,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:10,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:37:10,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:37:10,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 14:37:10,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:10,208::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432630|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432630 (Thu Nov 14 14:37:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:37:10,209::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:37:10,217::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:37:10,218::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:37:10,218::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432630, 'last-update-local-ts': 1384432630.2175691, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:10,218::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:37:10,218::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432624, 'last-update-local-ts': 1384432630.2175691, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:10,218::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:37:20,307::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:37:20,562::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:37:20,654::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:37:20,722::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:37:20,722::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:37:20,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:37:20,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:20,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:37:20,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:37:20,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 14:37:20,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:20,730::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432640|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432640 (Thu Nov 14 14:37:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:37:20,731::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:37:20,739::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:37:20,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:37:20,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432640, 'last-update-local-ts': 1384432640.739563, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:20,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:37:20,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432635, 'last-update-local-ts': 1384432640.739563, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:20,740::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:37:30,836::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:37:31,091::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:37:31,175::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:37:31,235::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:37:31,235::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:37:31,236::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:37:31,236::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:31,236::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:37:31,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:37:31,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 14:37:31,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:31,238::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432651|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432651 (Thu Nov 14 14:37:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:37:31,238::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:37:31,247::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:37:31,248::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:37:31,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432651, 'last-update-local-ts': 1384432651.2476161, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:31,248::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:37:31,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432645, 'last-update-local-ts': 1384432651.2476161, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:31,248::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:37:41,323::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:37:41,689::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:37:41,762::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:37:41,822::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:37:41,822::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:37:41,823::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:37:41,823::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:41,823::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:37:41,824::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:37:41,824::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.345 >MainThread::INFO::2013-11-14 14:37:41,824::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:41,825::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432661|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432661 (Thu Nov 14 14:37:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.345 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:37:41,825::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:37:41,831::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:37:41,832::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:37:41,832::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432661, 'last-update-local-ts': 1384432661.8317499, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:41,832::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:37:41,832::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432656, 'last-update-local-ts': 1384432661.8317499, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:41,833::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:37:51,918::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:37:52,171::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:37:52,264::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:37:52,474::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:37:52,475::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:37:52,475::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:37:52,475::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:52,476::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:37:52,476::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:37:52,476::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.29 >MainThread::INFO::2013-11-14 14:37:52,476::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:37:52,477::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432672|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432672 (Thu Nov 14 14:37:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.29 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:37:52,477::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:37:52,485::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:37:52,486::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:37:52,486::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432672, 'last-update-local-ts': 1384432672.4857709, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:52,486::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:37:52,486::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432666, 'last-update-local-ts': 1384432672.4857709, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:37:52,487::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:38:02,582::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:38:02,836::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:38:02,909::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:38:02,970::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:38:02,970::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:38:02,971::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:38:02,971::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:02,971::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:38:02,971::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:38:02,972::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.245 >MainThread::INFO::2013-11-14 14:38:02,972::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:02,973::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432682|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432682 (Thu Nov 14 14:38:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.245 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:38:02,973::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:38:02,982::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:38:02,982::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:38:02,982::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432682, 'last-update-local-ts': 1384432682.9820399, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:02,982::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:38:02,983::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432677, 'last-update-local-ts': 1384432682.9820399, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:02,983::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:38:13,065::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:38:13,338::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:38:13,434::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:38:13,496::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:38:13,496::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:38:13,503::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:38:13,503::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:13,503::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:38:13,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:38:13,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 14:38:13,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:13,505::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432693|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432693 (Thu Nov 14 14:38:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:38:13,505::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:38:13,513::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:38:13,513::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:38:13,513::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432693, 'last-update-local-ts': 1384432693.513133, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:13,513::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:38:13,514::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432687, 'last-update-local-ts': 1384432693.513133, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:13,514::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:38:23,594::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:38:23,850::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:38:23,922::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:38:23,983::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:38:23,984::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:38:23,984::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:38:23,984::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:23,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:38:23,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:38:23,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.62 >MainThread::INFO::2013-11-14 14:38:23,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:23,986::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432703|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432703 (Thu Nov 14 14:38:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.62 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:38:23,987::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:38:23,996::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:38:23,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:38:23,996::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432703, 'last-update-local-ts': 1384432703.9960589, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:23,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:38:23,997::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432698, 'last-update-local-ts': 1384432703.9960589, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:23,997::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:38:34,071::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:38:34,326::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:38:34,384::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:38:34,445::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:38:34,445::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:38:34,446::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:38:34,446::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:34,446::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:38:34,446::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:38:34,447::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.525 >MainThread::INFO::2013-11-14 14:38:34,447::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:34,447::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432714|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432714 (Thu Nov 14 14:38:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.525 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:38:34,448::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:38:34,456::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:38:34,456::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:38:34,456::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432714, 'last-update-local-ts': 1384432714.4561241, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:34,456::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:38:34,457::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432708, 'last-update-local-ts': 1384432714.4561241, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:34,457::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:38:44,536::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:38:44,785::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:38:44,880::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:38:44,940::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:38:44,940::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:38:44,947::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:38:44,947::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:44,947::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:38:44,948::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:38:44,948::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.445 >MainThread::INFO::2013-11-14 14:38:44,948::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:44,949::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432724|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432724 (Thu Nov 14 14:38:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.445 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:38:44,949::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:38:44,958::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:38:44,959::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:38:44,959::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432724, 'last-update-local-ts': 1384432724.9586599, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:44,959::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:38:44,959::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432719, 'last-update-local-ts': 1384432724.9586599, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:44,959::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:38:55,035::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:38:55,286::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:38:55,367::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:38:55,427::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:38:55,428::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:38:55,428::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:38:55,428::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:55,429::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:38:55,429::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:38:55,429::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.375 >MainThread::INFO::2013-11-14 14:38:55,429::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:38:55,431::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432735|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432735 (Thu Nov 14 14:38:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.375 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:38:55,431::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:38:55,439::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:38:55,440::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:38:55,440::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432735, 'last-update-local-ts': 1384432735.4396739, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:55,440::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:38:55,440::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432729, 'last-update-local-ts': 1384432735.4396739, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:38:55,440::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:39:05,512::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:39:05,771::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:39:05,862::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:39:05,929::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:39:05,929::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:39:05,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:39:05,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:05,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:39:05,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:39:05,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.715 >MainThread::INFO::2013-11-14 14:39:05,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:05,931::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432745|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432745 (Thu Nov 14 14:39:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.715 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:39:05,932::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:39:05,941::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:39:05,941::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:39:05,942::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432745, 'last-update-local-ts': 1384432745.9413791, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:05,942::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:39:05,942::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432740, 'last-update-local-ts': 1384432745.9413791, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:05,942::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:39:16,025::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:39:16,279::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:39:16,363::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:39:16,423::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:39:16,424::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:39:16,430::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:39:16,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:16,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:39:16,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:39:16,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.605 >MainThread::INFO::2013-11-14 14:39:16,432::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:16,433::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432756|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432756 (Thu Nov 14 14:39:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.605 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:39:16,433::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:39:16,442::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:39:16,443::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:39:16,443::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432756, 'last-update-local-ts': 1384432756.4427769, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:16,443::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:39:16,443::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432750, 'last-update-local-ts': 1384432756.4427769, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:16,444::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:39:26,519::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:39:26,769::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:39:26,831::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:39:26,892::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:39:26,892::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:39:26,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:39:26,893::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:26,893::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:39:26,893::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:39:26,893::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.51 >MainThread::INFO::2013-11-14 14:39:26,894::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:26,894::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432766|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432766 (Thu Nov 14 14:39:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.51 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:39:26,894::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:39:26,903::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:39:26,903::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:39:26,903::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432766, 'last-update-local-ts': 1384432766.9031799, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:26,904::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:39:26,904::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432761, 'last-update-local-ts': 1384432766.9031799, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:26,904::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:39:36,972::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:39:37,239::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:39:37,322::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:39:37,399::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:39:37,399::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:39:37,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:39:37,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:37,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:39:37,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:39:37,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.875 >MainThread::INFO::2013-11-14 14:39:37,401::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:37,401::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432777|1|2300|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432777 (Thu Nov 14 14:39:37 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.875 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:39:37,401::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:39:37,410::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:39:37,410::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:39:37,410::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432777, 'last-update-local-ts': 1384432777.4102299, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:37,411::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:39:37,411::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432771, 'last-update-local-ts': 1384432777.4102299, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:37,411::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:39:47,497::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:39:47,753::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:39:47,853::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:39:47,914::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:39:47,914::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:39:47,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:39:47,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:47,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:39:47,916::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:39:47,916::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.74 >MainThread::INFO::2013-11-14 14:39:47,916::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:47,917::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432787|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432787 (Thu Nov 14 14:39:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.74 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:39:47,917::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:39:47,926::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:39:47,926::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:39:47,926::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432787, 'last-update-local-ts': 1384432787.926306, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:47,927::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:39:47,927::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432781, 'last-update-local-ts': 1384432787.926306, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:47,927::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:39:58,006::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:39:58,256::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:39:58,346::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:39:58,408::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:39:58,408::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:39:58,409::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:39:58,409::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:58,409::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:39:58,409::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:39:58,410::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.625 >MainThread::INFO::2013-11-14 14:39:58,410::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:39:58,410::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432798|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432798 (Thu Nov 14 14:39:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.625 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:39:58,411::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:39:58,419::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:39:58,419::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:39:58,420::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432798, 'last-update-local-ts': 1384432798.419399, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:58,420::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:39:58,420::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432792, 'last-update-local-ts': 1384432798.419399, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:39:58,420::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:40:08,514::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:40:08,809::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:40:08,907::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:40:08,968::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:40:08,968::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:40:08,969::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:40:08,969::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:08,969::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:40:08,969::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:40:08,970::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.57 >MainThread::INFO::2013-11-14 14:40:08,970::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:08,971::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432808|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432808 (Thu Nov 14 14:40:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.57 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:40:08,971::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:40:08,978::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:40:08,978::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:40:08,979::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432808, 'last-update-local-ts': 1384432808.9784861, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:08,979::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:40:08,979::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432802, 'last-update-local-ts': 1384432808.9784861, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:08,979::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:40:19,051::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:40:19,306::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:40:19,376::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:40:19,437::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:40:19,437::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:40:19,437::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:40:19,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:19,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 14:40:19,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:40:19,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.555 >MainThread::INFO::2013-11-14 14:40:19,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:19,439::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432819|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432819 (Thu Nov 14 14:40:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.555 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:40:19,439::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:40:19,447::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:40:19,447::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:40:19,448::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432819, 'last-update-local-ts': 1384432819.4475369, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:19,448::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:40:19,448::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432813, 'last-update-local-ts': 1384432819.4475369, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:19,448::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:40:29,542::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:40:29,791::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:40:29,886::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:40:29,948::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:40:29,948::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:40:29,948::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:40:29,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:29,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:40:29,950::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:40:29,950::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.47 >MainThread::INFO::2013-11-14 14:40:29,950::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:29,951::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432829|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432829 (Thu Nov 14 14:40:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.47 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:40:29,952::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:40:29,966::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:40:29,966::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:40:29,966::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432829, 'last-update-local-ts': 1384432829.965868, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:29,966::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:40:29,967::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432823, 'last-update-local-ts': 1384432829.965868, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:29,967::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:40:40,042::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:40:40,296::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:40:40,384::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:40:40,451::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:40:40,451::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:40:40,457::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:40:40,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:40,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:40:40,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:40:40,459::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.395 >MainThread::INFO::2013-11-14 14:40:40,459::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:40,459::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432840|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432840 (Thu Nov 14 14:40:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.395 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:40:40,460::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:40:40,468::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:40:40,468::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:40:40,469::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432840, 'last-update-local-ts': 1384432840.468483, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:40,469::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:40:40,469::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432834, 'last-update-local-ts': 1384432840.468483, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:40,469::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:40:50,541::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:40:50,796::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:40:50,887::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:40:50,956::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:40:50,956::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:40:50,957::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:40:50,957::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:50,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:40:50,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:40:50,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 14:40:50,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:40:50,960::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432850|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432850 (Thu Nov 14 14:40:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:40:50,960::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:40:50,971::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:40:50,971::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:40:50,972::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432850, 'last-update-local-ts': 1384432850.971442, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:50,972::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:40:50,972::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432844, 'last-update-local-ts': 1384432850.971442, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:40:50,972::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:41:01,051::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:41:01,299::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:41:01,381::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:41:01,441::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:41:01,442::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:41:01,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:41:01,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:01,443::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:41:01,443::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:41:01,443::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 14:41:01,444::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:01,444::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432861|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432861 (Thu Nov 14 14:41:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:41:01,444::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:41:01,453::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:41:01,453::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:41:01,453::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432861, 'last-update-local-ts': 1384432861.4530039, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:01,453::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:41:01,454::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432855, 'last-update-local-ts': 1384432861.4530039, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:01,454::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:41:11,527::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:41:11,784::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:41:11,848::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:41:11,907::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:41:11,908::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:41:11,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:41:11,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:11,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:41:11,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:41:11,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 14:41:11,916::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:11,916::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432871|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432871 (Thu Nov 14 14:41:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:41:11,916::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:41:11,925::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:41:11,926::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:41:11,926::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432871, 'last-update-local-ts': 1384432871.9257469, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:11,926::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:41:11,926::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432865, 'last-update-local-ts': 1384432871.9257469, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:11,927::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:41:22,020::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:41:22,272::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:41:22,410::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:41:22,496::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:41:22,496::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:41:22,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:41:22,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:22,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:41:22,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:41:22,498::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 14:41:22,498::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:22,499::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432882|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432882 (Thu Nov 14 14:41:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:41:22,499::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:41:22,508::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:41:22,508::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:41:22,509::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432882, 'last-update-local-ts': 1384432882.508338, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:22,509::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:41:22,509::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432876, 'last-update-local-ts': 1384432882.508338, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:22,509::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:41:32,586::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:41:32,839::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:41:32,937::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:41:33,002::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:41:33,002::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:41:33,009::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:41:33,009::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:33,009::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:41:33,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:41:33,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 14:41:33,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:33,011::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432893|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432893 (Thu Nov 14 14:41:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:41:33,011::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:41:33,020::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:41:33,020::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:41:33,020::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432893, 'last-update-local-ts': 1384432893.020241, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:33,021::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:41:33,021::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432886, 'last-update-local-ts': 1384432893.020241, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:33,021::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:41:43,100::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:41:43,374::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:41:43,456::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:41:43,525::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:41:43,525::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:41:43,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:41:43,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:43,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:41:43,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:41:43,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 14:41:43,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:43,529::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432903|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432903 (Thu Nov 14 14:41:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:41:43,529::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:41:43,538::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:41:43,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:41:43,539::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432903, 'last-update-local-ts': 1384432903.5384581, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:43,539::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:41:43,539::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432897, 'last-update-local-ts': 1384432903.5384581, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:43,539::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:41:53,625::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:41:53,885::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:41:53,970::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:41:54,043::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:41:54,043::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:41:54,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:41:54,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:54,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 14:41:54,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:41:54,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 14:41:54,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:41:54,045::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432914|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432914 (Thu Nov 14 14:41:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:41:54,045::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:41:54,053::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:41:54,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:41:54,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432914, 'last-update-local-ts': 1384432914.0538831, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:54,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:41:54,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432907, 'last-update-local-ts': 1384432914.0538831, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:41:54,055::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:42:04,127::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:42:04,382::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:42:04,444::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:42:04,503::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:42:04,503::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:42:04,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:42:04,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:04,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:42:04,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:42:04,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.195 >MainThread::INFO::2013-11-14 14:42:04,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:04,506::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432924|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432924 (Thu Nov 14 14:42:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.195 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:42:04,506::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:42:04,516::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:42:04,516::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:42:04,516::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432924, 'last-update-local-ts': 1384432924.516247, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:04,517::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:42:04,517::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432917, 'last-update-local-ts': 1384432924.516247, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:04,517::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:42:14,616::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:42:14,865::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:42:14,949::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:42:15,010::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:42:15,010::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:42:15,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:42:15,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:15,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:42:15,012::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:42:15,012::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.165 >MainThread::INFO::2013-11-14 14:42:15,012::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:15,013::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432935|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432935 (Thu Nov 14 14:42:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.165 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:42:15,013::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:42:15,022::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:42:15,022::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:42:15,022::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432935, 'last-update-local-ts': 1384432935.0219769, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:15,022::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:42:15,023::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432928, 'last-update-local-ts': 1384432935.0219769, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:15,023::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:42:25,103::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:42:25,350::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:42:25,437::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:42:25,497::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:42:25,498::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:42:25,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:42:25,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:25,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:42:25,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:42:25,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 14:42:25,506::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:25,506::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432945|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432945 (Thu Nov 14 14:42:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:42:25,506::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:42:25,516::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:42:25,516::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:42:25,517::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432945, 'last-update-local-ts': 1384432945.5164659, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:25,517::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:42:25,517::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432938, 'last-update-local-ts': 1384432945.5164659, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:25,517::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:42:35,603::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:42:35,860::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:42:35,947::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:42:36,009::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:42:36,009::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:42:36,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:42:36,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:36,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:42:36,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:42:36,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.48 >MainThread::INFO::2013-11-14 14:42:36,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:36,013::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432956|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432956 (Thu Nov 14 14:42:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.48 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:42:36,013::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:42:36,022::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:42:36,022::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:42:36,022::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432956, 'last-update-local-ts': 1384432956.0222681, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:36,023::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:42:36,023::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432949, 'last-update-local-ts': 1384432956.0222681, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:36,023::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:42:46,106::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:42:46,365::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:42:46,448::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:42:46,516::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:42:46,516::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:42:46,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:42:46,517::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:46,517::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:42:46,517::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:42:46,518::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.405 >MainThread::INFO::2013-11-14 14:42:46,518::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:46,518::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432966|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432966 (Thu Nov 14 14:42:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.405 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:42:46,518::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:42:46,527::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:42:46,527::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:42:46,527::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432966, 'last-update-local-ts': 1384432966.5271001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:46,527::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:42:46,528::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432959, 'last-update-local-ts': 1384432966.5271001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:46,528::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:42:56,594::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:42:56,852::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:42:56,919::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:42:56,978::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:42:56,978::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:42:56,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:42:56,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:56,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:42:56,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:42:56,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.34 >MainThread::INFO::2013-11-14 14:42:56,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:42:56,987::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432976|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432976 (Thu Nov 14 14:42:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.34 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:42:56,987::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:42:56,997::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:42:56,997::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:42:56,997::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432976, 'last-update-local-ts': 1384432976.9969549, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:56,997::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:42:56,997::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432970, 'last-update-local-ts': 1384432976.9969549, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:42:56,998::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:43:07,084::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:43:07,335::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:43:07,433::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:43:07,492::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:43:07,493::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:43:07,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:43:07,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:07,494::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:43:07,494::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:43:07,494::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.33 >MainThread::INFO::2013-11-14 14:43:07,495::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:07,496::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432987|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432987 (Thu Nov 14 14:43:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.33 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:43:07,496::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:43:07,504::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:43:07,504::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:43:07,504::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432987, 'last-update-local-ts': 1384432987.5038929, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:07,504::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:43:07,504::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432980, 'last-update-local-ts': 1384432987.5038929, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:07,505::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:43:17,580::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:43:17,834::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:43:17,923::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:43:17,999::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:43:17,999::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:43:18,000::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:43:18,000::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:18,000::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:43:18,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:43:18,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.28 >MainThread::INFO::2013-11-14 14:43:18,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:18,002::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384432998|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384432998 (Thu Nov 14 14:43:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.28 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:43:18,002::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:43:18,010::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:43:18,011::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:43:18,011::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384432998, 'last-update-local-ts': 1384432998.0107501, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:18,011::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:43:18,011::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384432991, 'last-update-local-ts': 1384432998.0107501, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:18,012::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:43:28,101::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:43:28,360::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:43:28,465::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:43:28,525::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:43:28,526::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:43:28,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:43:28,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:28,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:43:28,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:43:28,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 14:43:28,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:28,529::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433008|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433008 (Thu Nov 14 14:43:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:43:28,529::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:43:28,537::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:43:28,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:43:28,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433008, 'last-update-local-ts': 1384433008.537679, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:28,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:43:28,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433001, 'last-update-local-ts': 1384433008.537679, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:28,539::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:43:38,607::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:43:38,918::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:43:38,984::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:43:39,043::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:43:39,043::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:43:39,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:43:39,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:39,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:43:39,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:43:39,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 14:43:39,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:39,046::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433019|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433019 (Thu Nov 14 14:43:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:43:39,046::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:43:39,053::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:43:39,053::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:43:39,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433019, 'last-update-local-ts': 1384433019.0533881, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:39,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:43:39,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433012, 'last-update-local-ts': 1384433019.0533881, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:39,054::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:43:49,146::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:43:49,396::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:43:49,481::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:43:49,543::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:43:49,544::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:43:49,544::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:43:49,544::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:49,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:43:49,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:43:49,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 14:43:49,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:43:49,546::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433029|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433029 (Thu Nov 14 14:43:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:43:49,546::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:43:49,554::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:43:49,555::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:43:49,555::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433029, 'last-update-local-ts': 1384433029.554811, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:49,555::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:43:49,555::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433022, 'last-update-local-ts': 1384433029.554811, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:43:49,556::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:43:59,637::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:43:59,885::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:43:59,972::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:44:00,038::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:44:00,039::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:44:00,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:44:00,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:00,046::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:44:00,046::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:44:00,046::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 14:44:00,046::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:00,047::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433040|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433040 (Thu Nov 14 14:44:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:44:00,047::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:44:00,057::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:44:00,057::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:44:00,058::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433040, 'last-update-local-ts': 1384433040.057441, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:00,058::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:44:00,058::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433033, 'last-update-local-ts': 1384433040.057441, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:00,058::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:44:10,174::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:44:10,430::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:44:10,519::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:44:10,579::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:44:10,579::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:44:10,580::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:44:10,580::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:10,580::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:44:10,581::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:44:10,581::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 14:44:10,581::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:10,582::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433050|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433050 (Thu Nov 14 14:44:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:44:10,582::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:44:10,589::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:44:10,589::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:44:10,590::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433050, 'last-update-local-ts': 1384433050.5893519, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:10,590::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:44:10,590::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433043, 'last-update-local-ts': 1384433050.5893519, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:10,590::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:44:20,664::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:44:20,919::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:44:20,992::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:44:21,052::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:44:21,052::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:44:21,053::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:44:21,053::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:21,053::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7062 >MainThread::INFO::2013-11-14 14:44:21,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:44:21,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 14:44:21,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:21,054::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433061|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433061 (Thu Nov 14 14:44:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7062 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:44:21,055::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:44:21,063::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:44:21,064::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:44:21,064::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433061, 'last-update-local-ts': 1384433061.063602, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:21,064::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:44:21,064::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433054, 'last-update-local-ts': 1384433061.063602, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:21,064::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:44:31,151::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:44:31,406::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:44:31,488::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:44:31,548::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:44:31,548::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:44:31,548::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:44:31,549::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:31,549::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:44:31,549::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:44:31,549::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 14:44:31,550::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:31,550::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433071|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433071 (Thu Nov 14 14:44:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:44:31,550::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:44:31,558::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:44:31,558::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:44:31,558::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433071, 'last-update-local-ts': 1384433071.558069, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:31,558::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:44:31,559::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433064, 'last-update-local-ts': 1384433071.558069, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:31,559::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:44:41,648::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:44:41,897::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:44:41,992::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:44:42,053::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:44:42,053::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:44:42,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:44:42,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:42,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:44:42,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:44:42,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 14:44:42,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:42,056::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433082|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433082 (Thu Nov 14 14:44:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:44:42,057::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:44:42,065::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:44:42,066::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:44:42,066::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433082, 'last-update-local-ts': 1384433082.0656481, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:42,066::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:44:42,066::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433074, 'last-update-local-ts': 1384433082.0656481, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:42,066::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:44:52,133::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:44:52,399::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:44:52,490::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:44:52,550::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:44:52,551::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:44:52,557::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:44:52,557::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:52,558::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:44:52,558::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:44:52,558::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 14:44:52,559::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:44:52,559::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433092|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433092 (Thu Nov 14 14:44:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:44:52,559::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:44:52,568::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:44:52,568::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:44:52,569::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433092, 'last-update-local-ts': 1384433092.5685289, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:52,569::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:44:52,569::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433085, 'last-update-local-ts': 1384433092.5685289, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:44:52,569::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:45:02,651::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:45:02,904::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:45:02,983::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:45:03,044::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:45:03,045::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:45:03,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:45:03,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:03,046::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:45:03,046::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:45:03,046::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 14:45:03,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:03,047::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433103|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433103 (Thu Nov 14 14:45:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:45:03,048::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:45:03,055::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:45:03,055::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:45:03,056::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433103, 'last-update-local-ts': 1384433103.0555301, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:03,056::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:45:03,056::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433095, 'last-update-local-ts': 1384433103.0555301, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:03,056::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:45:13,127::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:45:13,479::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:45:13,547::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:45:13,608::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:45:13,609::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:45:13,609::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:45:13,609::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:13,610::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:45:13,610::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:45:13,610::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 14:45:13,610::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:13,611::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433113|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433113 (Thu Nov 14 14:45:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:45:13,611::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:45:13,619::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:45:13,620::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:45:13,620::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433113, 'last-update-local-ts': 1384433113.6196799, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:13,620::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:45:13,620::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433106, 'last-update-local-ts': 1384433113.6196799, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:13,620::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:45:23,706::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:45:23,958::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:45:24,039::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:45:24,100::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:45:24,100::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:45:24,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:45:24,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:24,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:45:24,108::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:45:24,108::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 14:45:24,108::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:24,109::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433124|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433124 (Thu Nov 14 14:45:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:45:24,109::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:45:24,117::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:45:24,117::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:45:24,117::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433124, 'last-update-local-ts': 1384433124.1171629, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:24,118::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:45:24,118::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433116, 'last-update-local-ts': 1384433124.1171629, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:24,118::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:45:34,207::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:45:34,587::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:45:34,699::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:45:34,759::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:45:34,759::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:45:34,760::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:45:34,760::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:34,760::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:45:34,760::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:45:34,761::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 14:45:34,761::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:34,762::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433134|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433134 (Thu Nov 14 14:45:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:45:34,762::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:45:34,771::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:45:34,771::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:45:34,771::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433134, 'last-update-local-ts': 1384433134.771102, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:34,771::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:45:34,772::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433127, 'last-update-local-ts': 1384433134.771102, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:34,772::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:45:44,848::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:45:45,518::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:45:45,604::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:45:45,669::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:45:45,670::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:45:45,670::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:45:45,670::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:45,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7061 >MainThread::INFO::2013-11-14 14:45:45,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:45:45,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 14:45:45,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:45,672::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433145|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433145 (Thu Nov 14 14:45:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7061 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:45:45,672::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:45:45,681::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:45:45,681::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:45:45,681::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433145, 'last-update-local-ts': 1384433145.6810081, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:45,681::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:45:45,682::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433137, 'last-update-local-ts': 1384433145.6810081, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:45,682::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:45:55,748::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:45:56,005::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:45:56,095::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:45:56,156::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:45:56,156::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:45:56,157::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:45:56,157::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:56,157::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:45:56,157::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:45:56,158::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.56 >MainThread::INFO::2013-11-14 14:45:56,158::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:45:56,159::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433156|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433156 (Thu Nov 14 14:45:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.56 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:45:56,159::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:45:56,169::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:45:56,169::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:45:56,170::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433156, 'last-update-local-ts': 1384433156.1694019, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:56,170::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:45:56,170::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433148, 'last-update-local-ts': 1384433156.1694019, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:45:56,170::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:46:06,264::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:46:06,633::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:46:06,726::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:46:06,801::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:46:06,801::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:46:06,801::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:46:06,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:06,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:46:06,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:46:06,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.47 >MainThread::INFO::2013-11-14 14:46:06,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:06,803::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433166|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433166 (Thu Nov 14 14:46:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.47 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:46:06,803::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:46:06,812::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:46:06,812::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:46:06,812::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433166, 'last-update-local-ts': 1384433166.8120329, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:06,812::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:46:06,813::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433158, 'last-update-local-ts': 1384433166.8120329, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:06,813::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:46:16,901::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:46:17,159::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:46:17,217::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:46:17,277::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:46:17,277::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:46:17,277::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:46:17,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:17,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:46:17,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:46:17,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.4 >MainThread::INFO::2013-11-14 14:46:17,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:17,280::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433177|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433177 (Thu Nov 14 14:46:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.4 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:46:17,280::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:46:17,290::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:46:17,290::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:46:17,290::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433177, 'last-update-local-ts': 1384433177.290096, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:17,290::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:46:17,291::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433169, 'last-update-local-ts': 1384433177.290096, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:17,291::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:46:27,379::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:46:27,628::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:46:27,712::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:46:27,772::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:46:27,773::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:46:27,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:46:27,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:27,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:46:27,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:46:27,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.34 >MainThread::INFO::2013-11-14 14:46:27,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:27,775::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433187|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433187 (Thu Nov 14 14:46:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.34 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:46:27,775::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:46:27,784::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:46:27,784::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:46:27,784::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433187, 'last-update-local-ts': 1384433187.7839899, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:27,784::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:46:27,785::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433179, 'last-update-local-ts': 1384433187.7839899, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:27,785::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:46:37,860::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:46:38,124::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:46:38,224::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:46:38,284::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:46:38,284::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:46:38,291::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:46:38,291::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:38,291::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:46:38,292::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:46:38,292::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.765 >MainThread::INFO::2013-11-14 14:46:38,292::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:38,293::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433198|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433198 (Thu Nov 14 14:46:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.765 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:46:38,293::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:46:38,302::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:46:38,302::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:46:38,302::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433198, 'last-update-local-ts': 1384433198.3019979, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:38,302::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:46:38,303::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433190, 'last-update-local-ts': 1384433198.3019979, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:38,303::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:46:48,385::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:46:48,638::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:46:48,733::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:46:48,794::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:46:48,795::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:46:48,795::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:46:48,795::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:48,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:46:48,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:46:48,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.645 >MainThread::INFO::2013-11-14 14:46:48,797::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:48,798::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433208|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433208 (Thu Nov 14 14:46:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.645 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:46:48,798::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:46:48,813::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:46:48,813::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:46:48,813::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433208, 'last-update-local-ts': 1384433208.812942, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:48,813::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:46:48,814::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433211, 'last-update-local-ts': 1384433208.812942, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:48,814::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:46:58,891::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:46:59,182::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:46:59,252::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:46:59,313::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:46:59,313::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:46:59,314::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:46:59,314::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:59,314::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:46:59,314::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:46:59,315::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.625 >MainThread::INFO::2013-11-14 14:46:59,315::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:46:59,315::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433219|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433219 (Thu Nov 14 14:46:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.625 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:46:59,316::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:46:59,322::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:46:59,323::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:46:59,323::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433219, 'last-update-local-ts': 1384433219.322582, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:59,323::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:46:59,323::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433211, 'last-update-local-ts': 1384433208.812942, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:46:59,323::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:47:09,411::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:47:09,666::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:47:09,806::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:47:09,866::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:47:09,866::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:47:09,867::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:47:09,867::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:09,867::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:47:09,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:47:09,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.53 >MainThread::INFO::2013-11-14 14:47:09,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:09,869::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433229|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433229 (Thu Nov 14 14:47:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.53 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:47:09,869::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:47:09,879::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:47:09,879::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:47:09,879::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433229, 'last-update-local-ts': 1384433229.8791549, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:09,880::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:47:09,880::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433232, 'last-update-local-ts': 1384433229.8791549, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:09,880::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:47:19,982::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:47:20,235::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:47:20,324::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:47:20,385::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:47:20,385::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:47:20,386::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:47:20,386::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:20,386::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:47:20,386::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:47:20,387::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.89 >MainThread::INFO::2013-11-14 14:47:20,387::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:20,388::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433240|1|2300|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433240 (Thu Nov 14 14:47:20 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.89 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:47:20,388::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:47:20,397::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:47:20,398::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:47:20,398::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433240, 'last-update-local-ts': 1384433240.3975689, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:20,398::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:47:20,398::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433242, 'last-update-local-ts': 1384433240.3975689, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:20,398::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:47:30,480::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:47:30,728::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:47:30,823::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:47:30,884::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:47:30,884::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:47:30,891::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:47:30,891::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:30,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:47:30,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:47:30,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.755 >MainThread::INFO::2013-11-14 14:47:30,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:30,893::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433250|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433250 (Thu Nov 14 14:47:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.755 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:47:30,893::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:47:30,903::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:47:30,903::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:47:30,903::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433250, 'last-update-local-ts': 1384433250.9030759, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:30,903::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:47:30,904::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433253, 'last-update-local-ts': 1384433250.9030759, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:30,904::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:47:40,984::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:47:41,236::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:47:41,317::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:47:41,377::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:47:41,378::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:47:41,378::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:47:41,378::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:41,379::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:47:41,379::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:47:41,379::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.715 >MainThread::INFO::2013-11-14 14:47:41,379::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:41,380::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433261|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433261 (Thu Nov 14 14:47:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.715 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:47:41,381::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:47:41,389::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:47:41,389::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:47:41,390::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433261, 'last-update-local-ts': 1384433261.389411, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:41,390::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:47:41,390::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433263, 'last-update-local-ts': 1384433261.389411, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:41,390::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:47:51,464::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:47:51,716::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:47:51,795::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:47:51,856::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:47:51,857::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:47:51,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:47:51,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:51,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7061 >MainThread::INFO::2013-11-14 14:47:51,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:47:51,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.605 >MainThread::INFO::2013-11-14 14:47:51,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:47:51,859::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433271|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433271 (Thu Nov 14 14:47:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.605 >engine-health=vm-down >gateway=True >mem-free=7061 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:47:51,859::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:47:51,867::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:47:51,868::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:47:51,868::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433271, 'last-update-local-ts': 1384433271.8678679, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:51,868::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:47:51,868::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433274, 'last-update-local-ts': 1384433271.8678679, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:47:51,869::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:48:01,940::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:48:02,195::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:48:02,259::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:48:02,318::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:48:02,318::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:48:02,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:48:02,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:02,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:48:02,326::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:48:02,326::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.515 >MainThread::INFO::2013-11-14 14:48:02,326::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:02,327::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433282|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433282 (Thu Nov 14 14:48:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.515 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:48:02,327::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:48:02,337::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:48:02,337::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:48:02,338::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433282, 'last-update-local-ts': 1384433282.3374839, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:02,338::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:48:02,338::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433284, 'last-update-local-ts': 1384433282.3374839, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:02,338::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:48:12,421::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:48:12,680::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:48:12,776::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:48:12,839::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:48:12,839::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:48:12,839::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:48:12,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:12,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:48:12,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:48:12,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.47 >MainThread::INFO::2013-11-14 14:48:12,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:12,842::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433292|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433292 (Thu Nov 14 14:48:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.47 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:48:12,842::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:48:12,851::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:48:12,851::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:48:12,851::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433292, 'last-update-local-ts': 1384433292.8509951, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:12,851::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:48:12,852::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433295, 'last-update-local-ts': 1384433292.8509951, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:12,852::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:48:22,923::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:48:23,182::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:48:23,269::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:48:23,330::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:48:23,330::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:48:23,331::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:48:23,331::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:23,331::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:48:23,331::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:48:23,332::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.4 >MainThread::INFO::2013-11-14 14:48:23,332::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:23,332::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433303|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433303 (Thu Nov 14 14:48:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.4 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:48:23,333::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:48:23,341::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:48:23,341::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:48:23,342::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433303, 'last-update-local-ts': 1384433303.3415029, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:23,342::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:48:23,342::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433305, 'last-update-local-ts': 1384433303.3415029, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:23,342::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:48:33,418::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:48:33,691::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:48:33,791::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:48:33,855::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:48:33,856::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:48:33,856::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:48:33,856::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:33,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:48:33,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:48:33,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.415 >MainThread::INFO::2013-11-14 14:48:33,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:33,859::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433313|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433313 (Thu Nov 14 14:48:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.415 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:48:33,859::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:48:33,869::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:48:33,870::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:48:33,870::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433313, 'last-update-local-ts': 1384433313.869524, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:33,870::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:48:33,870::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433315, 'last-update-local-ts': 1384433313.869524, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:33,871::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:48:43,947::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:48:44,196::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:48:44,275::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:48:44,338::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:48:44,338::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:48:44,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:48:44,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:44,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:48:44,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:48:44,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.35 >MainThread::INFO::2013-11-14 14:48:44,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:44,341::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433324|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433324 (Thu Nov 14 14:48:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.35 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:48:44,341::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:48:44,349::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:48:44,350::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:48:44,350::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433324, 'last-update-local-ts': 1384433324.349575, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:44,350::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:48:44,350::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433326, 'last-update-local-ts': 1384433324.349575, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:44,350::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:48:54,431::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:48:54,686::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:48:54,770::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:48:54,830::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:48:54,831::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:48:54,837::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:48:54,837::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:54,838::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:48:54,838::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:48:54,838::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 14:48:54,838::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:48:54,839::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433334|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433334 (Thu Nov 14 14:48:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:48:54,839::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:48:54,849::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:48:54,849::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:48:54,849::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433334, 'last-update-local-ts': 1384433334.849072, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:54,849::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:48:54,850::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433336, 'last-update-local-ts': 1384433334.849072, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:48:54,850::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:49:04,934::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:49:05,184::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:49:05,240::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:49:05,299::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:49:05,299::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:49:05,300::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:49:05,300::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:05,300::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:49:05,300::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:49:05,301::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.25 >MainThread::INFO::2013-11-14 14:49:05,301::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:05,302::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433345|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433345 (Thu Nov 14 14:49:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.25 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:49:05,302::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:49:05,311::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:49:05,311::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:49:05,312::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433345, 'last-update-local-ts': 1384433345.311476, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:05,312::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:49:05,312::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433347, 'last-update-local-ts': 1384433345.311476, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:05,312::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:49:15,408::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:49:15,724::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:49:15,822::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:49:15,926::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:49:15,926::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:49:15,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:49:15,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:15,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:49:15,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:49:15,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 14:49:15,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:15,929::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433355|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433355 (Thu Nov 14 14:49:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:49:15,929::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:49:15,937::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:49:15,938::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:49:15,938::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433355, 'last-update-local-ts': 1384433355.9375341, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:15,938::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:49:15,938::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433357, 'last-update-local-ts': 1384433355.9375341, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:15,938::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:49:26,023::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:49:26,279::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:49:26,367::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:49:26,432::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:49:26,432::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:49:26,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:49:26,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:26,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:49:26,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:49:26,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.73 >MainThread::INFO::2013-11-14 14:49:26,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:26,435::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433366|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433366 (Thu Nov 14 14:49:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.73 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:49:26,435::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:49:26,443::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:49:26,443::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:49:26,443::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433366, 'last-update-local-ts': 1384433366.4432311, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:26,444::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:49:26,444::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433368, 'last-update-local-ts': 1384433366.4432311, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:26,444::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:49:36,540::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:49:36,788::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:49:36,874::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:49:36,934::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:49:36,934::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:49:36,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:49:36,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:36,936::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:49:36,936::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:49:36,936::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.615 >MainThread::INFO::2013-11-14 14:49:36,936::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:36,937::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433376|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433376 (Thu Nov 14 14:49:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.615 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:49:36,938::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:49:36,945::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:49:36,946::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:49:36,946::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433376, 'last-update-local-ts': 1384433376.9455609, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:36,946::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:49:36,946::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433378, 'last-update-local-ts': 1384433376.9455609, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:36,946::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:49:47,017::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:49:47,270::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:49:47,353::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:49:47,426::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:49:47,426::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:49:47,427::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:49:47,427::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:47,427::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:49:47,428::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:49:47,428::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.52 >MainThread::INFO::2013-11-14 14:49:47,428::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:47,429::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433387|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433387 (Thu Nov 14 14:49:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.52 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:49:47,429::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:49:47,437::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:49:47,438::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:49:47,438::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433387, 'last-update-local-ts': 1384433387.437602, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:47,438::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:49:47,438::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433389, 'last-update-local-ts': 1384433387.437602, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:47,438::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:49:57,508::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:49:57,760::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:49:57,832::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:49:57,896::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:49:57,896::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:49:57,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:49:57,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:57,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:49:57,900::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:49:57,900::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.92 >MainThread::INFO::2013-11-14 14:49:57,900::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:49:57,901::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433397|1|2300|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433397 (Thu Nov 14 14:49:57 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.92 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:49:57,901::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:49:57,919::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:49:57,919::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:49:57,920::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433397, 'last-update-local-ts': 1384433397.9194219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:57,926::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:49:57,926::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433399, 'last-update-local-ts': 1384433397.9194219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:49:57,928::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:50:08,041::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:50:08,337::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:50:08,444::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:50:08,525::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:50:08,525::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:50:08,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:50:08,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:08,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:50:08,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:50:08,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.78 >MainThread::INFO::2013-11-14 14:50:08,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:08,529::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433408|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433408 (Thu Nov 14 14:50:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.78 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:50:08,529::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:50:08,539::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:50:08,540::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:50:08,540::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433408, 'last-update-local-ts': 1384433408.539854, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:08,540::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:50:08,540::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433410, 'last-update-local-ts': 1384433408.539854, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:08,541::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:50:18,622::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:50:18,886::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:50:18,985::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:50:19,046::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:50:19,047::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:50:19,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:50:19,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:19,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7062 >MainThread::INFO::2013-11-14 14:50:19,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:50:19,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.66 >MainThread::INFO::2013-11-14 14:50:19,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:19,049::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433419|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433419 (Thu Nov 14 14:50:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.66 >engine-health=vm-down >gateway=True >mem-free=7062 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:50:19,049::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:50:19,056::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:50:19,056::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:50:19,056::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433419, 'last-update-local-ts': 1384433419.056087, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:19,056::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:50:19,057::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433420, 'last-update-local-ts': 1384433419.056087, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:19,057::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:50:29,177::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:50:29,437::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:50:29,533::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:50:29,593::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:50:29,593::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:50:29,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:50:29,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:29,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:50:29,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:50:29,595::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.56 >MainThread::INFO::2013-11-14 14:50:29,595::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:29,596::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433429|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433429 (Thu Nov 14 14:50:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.56 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:50:29,596::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:50:29,603::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:50:29,603::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:50:29,604::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433429, 'last-update-local-ts': 1384433429.6033189, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:29,604::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:50:29,604::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433430, 'last-update-local-ts': 1384433429.6033189, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:29,604::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:50:39,681::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:50:39,945::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:50:40,033::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:50:40,099::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:50:40,100::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:50:40,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:50:40,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:40,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:50:40,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:50:40,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.47 >MainThread::INFO::2013-11-14 14:50:40,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:40,102::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433440|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433440 (Thu Nov 14 14:50:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.47 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:50:40,102::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:50:40,109::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:50:40,109::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:50:40,109::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433440, 'last-update-local-ts': 1384433440.1091909, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:40,110::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:50:40,110::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433441, 'last-update-local-ts': 1384433440.1091909, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:40,110::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:50:50,188::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:50:50,438::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:50:50,530::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:50:50,592::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:50:50,593::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:50:50,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:50:50,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:50,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:50:50,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:50:50,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.4 >MainThread::INFO::2013-11-14 14:50:50,595::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:50:50,595::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433450|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433450 (Thu Nov 14 14:50:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.4 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:50:50,596::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:50:50,604::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:50:50,605::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:50:50,605::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433450, 'last-update-local-ts': 1384433450.60482, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:50,605::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:50:50,605::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433451, 'last-update-local-ts': 1384433450.60482, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:50:50,606::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:51:00,686::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:51:00,946::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:51:01,033::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:51:01,093::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:51:01,094::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:51:01,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:51:01,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:01,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:51:01,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:51:01,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.34 >MainThread::INFO::2013-11-14 14:51:01,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:01,096::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433461|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433461 (Thu Nov 14 14:51:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.34 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:51:01,096::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:51:01,104::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:51:01,104::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:51:01,105::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433461, 'last-update-local-ts': 1384433461.1044469, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:01,105::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:51:01,105::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433462, 'last-update-local-ts': 1384433461.1044469, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:01,105::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:51:11,172::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:51:11,430::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:51:11,495::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:51:11,555::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:51:11,555::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:51:11,562::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:51:11,562::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:11,562::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:51:11,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:51:11,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 14:51:11,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:11,564::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433471|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433471 (Thu Nov 14 14:51:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.32 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:51:11,564::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:51:11,574::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:51:11,574::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:51:11,574::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433471, 'last-update-local-ts': 1384433471.5742841, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:11,575::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:51:11,575::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433472, 'last-update-local-ts': 1384433471.5742841, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:11,575::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:51:21,663::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:51:21,910::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:51:22,016::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:51:22,076::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:51:22,077::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:51:22,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:51:22,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:22,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:51:22,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:51:22,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 14:51:22,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:22,080::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433482|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433482 (Thu Nov 14 14:51:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:51:22,080::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:51:22,088::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:51:22,089::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:51:22,089::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433482, 'last-update-local-ts': 1384433482.088567, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:22,089::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:51:22,089::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433483, 'last-update-local-ts': 1384433482.088567, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:22,089::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:51:32,166::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:51:32,416::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:51:32,505::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:51:32,571::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:51:32,571::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:51:32,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:51:32,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:32,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:51:32,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:51:32,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.26 >MainThread::INFO::2013-11-14 14:51:32,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:32,573::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433492|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433492 (Thu Nov 14 14:51:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.26 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:51:32,574::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:51:32,582::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:51:32,582::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:51:32,583::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433492, 'last-update-local-ts': 1384433492.58236, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:32,583::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:51:32,583::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433493, 'last-update-local-ts': 1384433492.58236, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:32,583::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:51:42,659::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:51:42,912::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:51:43,001::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:51:43,066::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:51:43,067::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:51:43,067::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:51:43,067::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:43,068::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:51:43,068::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:51:43,068::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.22 >MainThread::INFO::2013-11-14 14:51:43,068::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:43,070::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433503|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433503 (Thu Nov 14 14:51:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.22 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:51:43,070::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:51:43,079::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:51:43,079::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:51:43,080::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433503, 'last-update-local-ts': 1384433503.0794189, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:43,080::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:51:43,080::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433504, 'last-update-local-ts': 1384433503.0794189, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:43,080::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:51:53,152::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:51:53,405::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:51:53,486::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:51:53,546::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:51:53,547::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:51:53,547::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:51:53,547::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:53,548::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:51:53,548::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:51:53,548::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 14:51:53,548::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:51:53,549::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433513|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433513 (Thu Nov 14 14:51:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:51:53,549::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:51:53,556::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:51:53,556::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:51:53,556::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433513, 'last-update-local-ts': 1384433513.5560889, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:53,556::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:51:53,557::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433514, 'last-update-local-ts': 1384433513.5560889, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:51:53,557::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:52:03,623::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:52:03,904::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:52:03,970::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:52:04,028::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:52:04,029::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:52:04,035::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:52:04,035::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:04,036::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:52:04,036::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:52:04,036::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 14:52:04,037::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:04,037::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433524|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433524 (Thu Nov 14 14:52:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:52:04,037::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:52:04,047::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:52:04,047::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:52:04,048::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433524, 'last-update-local-ts': 1384433524.0474241, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:04,048::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:52:04,048::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433525, 'last-update-local-ts': 1384433524.0474241, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:04,048::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:52:14,135::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:52:14,385::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:52:14,483::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:52:14,544::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:52:14,544::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:52:14,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:52:14,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:14,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:52:14,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:52:14,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 14:52:14,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:14,547::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433534|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433534 (Thu Nov 14 14:52:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:52:14,547::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:52:14,554::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:52:14,554::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:52:14,555::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433534, 'last-update-local-ts': 1384433534.5544679, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:14,555::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:52:14,555::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433535, 'last-update-local-ts': 1384433534.5544679, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:14,555::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:52:24,633::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:52:24,886::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:52:24,983::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:52:25,043::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:52:25,043::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:52:25,050::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:52:25,050::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:25,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:52:25,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:52:25,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 14:52:25,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:25,052::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433545|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433545 (Thu Nov 14 14:52:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:52:25,052::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:52:25,060::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:52:25,060::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:52:25,060::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433545, 'last-update-local-ts': 1384433545.060154, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:25,061::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:52:25,061::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433545, 'last-update-local-ts': 1384433545.060154, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:25,061::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:52:35,143::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:52:35,408::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:52:35,492::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:52:35,554::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:52:35,554::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:52:35,555::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:52:35,555::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:35,555::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:52:35,556::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:52:35,556::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 14:52:35,556::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:35,557::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433555|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433555 (Thu Nov 14 14:52:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:52:35,557::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:52:35,566::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:52:35,566::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:52:35,566::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433555, 'last-update-local-ts': 1384433555.5660081, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:35,566::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:52:35,567::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433556, 'last-update-local-ts': 1384433555.5660081, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:35,567::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:52:45,646::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:52:45,902::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:52:45,987::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:52:46,048::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:52:46,048::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:52:46,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:52:46,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:46,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:52:46,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:52:46,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 14:52:46,050::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:46,050::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433566|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433566 (Thu Nov 14 14:52:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:52:46,050::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:52:46,059::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:52:46,059::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:52:46,059::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433566, 'last-update-local-ts': 1384433566.059098, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:46,059::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:52:46,060::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433566, 'last-update-local-ts': 1384433566.059098, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:46,060::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:52:56,133::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:52:56,396::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:52:56,461::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:52:56,521::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:52:56,521::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:52:56,522::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:52:56,522::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:56,522::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:52:56,523::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:52:56,523::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 14:52:56,523::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:52:56,524::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433576|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433576 (Thu Nov 14 14:52:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:52:56,524::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:52:56,533::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:52:56,534::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:52:56,534::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433576, 'last-update-local-ts': 1384433576.533746, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:56,534::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:52:56,534::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433577, 'last-update-local-ts': 1384433576.533746, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:52:56,535::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:53:06,627::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:53:06,876::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:53:06,980::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:53:07,040::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:53:07,041::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:53:07,041::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:53:07,041::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:07,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 14:53:07,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:53:07,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 14:53:07,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:07,044::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433587|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433587 (Thu Nov 14 14:53:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:53:07,044::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:53:07,053::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:53:07,053::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:53:07,053::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433587, 'last-update-local-ts': 1384433587.05316, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:07,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:53:07,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433587, 'last-update-local-ts': 1384433587.05316, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:07,054::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:53:17,130::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:53:17,383::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:53:17,473::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:53:17,545::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:53:17,545::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:53:17,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:53:17,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:17,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:53:17,547::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:53:17,547::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 14:53:17,547::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:17,548::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433597|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433597 (Thu Nov 14 14:53:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:53:17,548::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:53:17,556::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:53:17,557::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:53:17,557::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433597, 'last-update-local-ts': 1384433597.5566449, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:17,557::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:53:17,557::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433598, 'last-update-local-ts': 1384433597.5566449, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:17,557::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:53:27,633::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:53:27,887::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:53:27,975::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:53:28,037::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:53:28,037::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:53:28,037::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:53:28,038::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:28,038::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:53:28,038::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:53:28,039::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 14:53:28,039::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:28,040::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433608|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433608 (Thu Nov 14 14:53:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:53:28,040::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:53:28,049::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:53:28,049::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:53:28,049::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433608, 'last-update-local-ts': 1384433608.0490911, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:28,049::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:53:28,050::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433608, 'last-update-local-ts': 1384433608.0490911, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:28,050::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:53:38,119::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:53:38,366::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:53:38,430::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:53:38,490::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:53:38,491::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:53:38,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:53:38,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:38,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:53:38,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:53:38,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 14:53:38,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:38,493::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433618|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433618 (Thu Nov 14 14:53:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:53:38,493::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:53:38,502::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:53:38,502::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:53:38,502::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433618, 'last-update-local-ts': 1384433618.5019879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:38,502::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:53:38,503::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433619, 'last-update-local-ts': 1384433618.5019879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:38,503::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:53:48,579::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:53:48,833::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:53:48,924::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:53:48,984::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:53:48,985::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:53:48,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:53:48,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:48,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:53:48,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:53:48,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 14:53:48,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:48,987::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433628|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433628 (Thu Nov 14 14:53:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:53:48,987::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:53:48,996::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:53:48,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:53:48,996::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433628, 'last-update-local-ts': 1384433628.995985, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:48,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:53:48,997::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433629, 'last-update-local-ts': 1384433628.995985, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:48,997::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:53:59,081::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:53:59,360::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:53:59,457::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:53:59,518::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:53:59,518::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:53:59,519::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:53:59,519::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:59,519::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7060 >MainThread::INFO::2013-11-14 14:53:59,520::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:53:59,520::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 14:53:59,520::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:53:59,521::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433639|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433639 (Thu Nov 14 14:53:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7060 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:53:59,522::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:53:59,528::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:53:59,528::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:53:59,529::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433639, 'last-update-local-ts': 1384433639.5284541, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:59,529::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:53:59,529::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433640, 'last-update-local-ts': 1384433639.5284541, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:53:59,529::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:54:09,605::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:54:09,857::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:54:09,948::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:54:10,015::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:54:10,015::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:54:10,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:54:10,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:10,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:54:10,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:54:10,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 14:54:10,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:10,018::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433650|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433650 (Thu Nov 14 14:54:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:54:10,018::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:54:10,026::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:54:10,026::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:54:10,027::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433650, 'last-update-local-ts': 1384433650.026386, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:10,027::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:54:10,027::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433650, 'last-update-local-ts': 1384433650.026386, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:10,027::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:54:20,093::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:54:20,368::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:54:20,530::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:54:20,595::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:54:20,596::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:54:20,596::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:54:20,596::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:20,597::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:54:20,597::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:54:20,597::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 14:54:20,598::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:20,599::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433660|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433660 (Thu Nov 14 14:54:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:54:20,599::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:54:20,607::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:54:20,608::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:54:20,608::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433660, 'last-update-local-ts': 1384433660.607861, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:20,608::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:54:20,608::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433660, 'last-update-local-ts': 1384433660.607861, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:20,609::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:54:30,698::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:54:30,989::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:54:31,065::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:54:31,125::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:54:31,126::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:54:31,126::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:54:31,126::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:31,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:54:31,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:54:31,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 14:54:31,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:31,128::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433671|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433671 (Thu Nov 14 14:54:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:54:31,128::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:54:31,136::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:54:31,137::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:54:31,137::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433671, 'last-update-local-ts': 1384433671.1367381, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:31,137::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:54:31,137::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433671, 'last-update-local-ts': 1384433671.1367381, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:31,138::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:54:41,209::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:54:41,463::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:54:41,532::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:54:41,592::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:54:41,592::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:54:41,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:54:41,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:41,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:54:41,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:54:41,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 14:54:41,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:41,594::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433681|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433681 (Thu Nov 14 14:54:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:54:41,595::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:54:41,603::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:54:41,603::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:54:41,604::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433681, 'last-update-local-ts': 1384433681.6034219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:41,604::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:54:41,604::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433681, 'last-update-local-ts': 1384433681.6034219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:41,604::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:54:51,684::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:54:51,943::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:54:52,032::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:54:52,091::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:54:52,092::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:54:52,098::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:54:52,098::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:52,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:54:52,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:54:52,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.25 >MainThread::INFO::2013-11-14 14:54:52,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:54:52,100::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433692|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433692 (Thu Nov 14 14:54:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.25 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:54:52,100::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:54:52,110::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:54:52,110::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:54:52,110::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433692, 'last-update-local-ts': 1384433692.1100211, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:52,110::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:54:52,111::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433692, 'last-update-local-ts': 1384433692.1100211, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:54:52,111::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:55:02,206::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:55:02,468::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:55:02,566::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:55:02,634::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:55:02,634::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:55:02,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:55:02,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:02,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:55:02,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:55:02,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 14:55:02,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:02,637::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433702|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433702 (Thu Nov 14 14:55:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:55:02,638::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:55:02,646::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:55:02,647::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:55:02,647::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433702, 'last-update-local-ts': 1384433702.6464739, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:02,647::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:55:02,647::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433703, 'last-update-local-ts': 1384433702.6464739, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:02,648::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:55:12,725::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:55:12,987::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:55:13,091::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:55:13,153::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:55:13,153::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:55:13,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:55:13,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:13,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:55:13,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:55:13,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 14:55:13,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:13,156::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433713|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433713 (Thu Nov 14 14:55:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:55:13,156::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:55:13,165::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:55:13,165::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:55:13,165::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433713, 'last-update-local-ts': 1384433713.1651211, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:13,165::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:55:13,166::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433713, 'last-update-local-ts': 1384433713.1651211, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:13,166::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:55:23,236::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:55:23,492::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:55:23,572::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:55:23,633::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:55:23,633::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:55:23,634::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:55:23,634::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:23,634::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:55:23,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:55:23,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.15 >MainThread::INFO::2013-11-14 14:55:23,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:23,635::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433723|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433723 (Thu Nov 14 14:55:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.15 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:55:23,636::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:55:23,644::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:55:23,644::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:55:23,644::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433723, 'last-update-local-ts': 1384433723.644264, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:23,645::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:55:23,645::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433724, 'last-update-local-ts': 1384433723.644264, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:23,645::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:55:33,717::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:55:33,968::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:55:34,053::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:55:34,113::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:55:34,114::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:55:34,120::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:55:34,120::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:34,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:55:34,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:55:34,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 14:55:34,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:34,122::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433734|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433734 (Thu Nov 14 14:55:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:55:34,122::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:55:34,132::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:55:34,132::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:55:34,132::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433734, 'last-update-local-ts': 1384433734.1320591, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:34,132::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:55:34,133::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433734, 'last-update-local-ts': 1384433734.1320591, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:34,133::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:55:44,218::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:55:44,468::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:55:44,552::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:55:44,613::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:55:44,613::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:55:44,614::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:55:44,614::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:44,614::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:55:44,615::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:55:44,615::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 14:55:44,615::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:44,616::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433744|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433744 (Thu Nov 14 14:55:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:55:44,616::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:55:44,625::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:55:44,625::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:55:44,626::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433744, 'last-update-local-ts': 1384433744.6254289, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:44,626::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:55:44,626::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433745, 'last-update-local-ts': 1384433744.6254289, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:44,626::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:55:54,698::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:55:54,946::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:55:55,028::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:55:55,089::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:55:55,090::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:55:55,090::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:55:55,090::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:55,091::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:55:55,091::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:55:55,091::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 14:55:55,092::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:55:55,092::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433755|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433755 (Thu Nov 14 14:55:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:55:55,092::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:55:55,101::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:55:55,101::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:55:55,101::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433755, 'last-update-local-ts': 1384433755.1009741, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:55,101::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:55:55,102::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433755, 'last-update-local-ts': 1384433755.1009741, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:55:55,102::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:56:05,172::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:56:05,425::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:56:05,498::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:56:05,556::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:56:05,557::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:56:05,557::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:56:05,557::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:05,558::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:56:05,558::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:56:05,558::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 14:56:05,559::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:05,559::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433765|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433765 (Thu Nov 14 14:56:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:56:05,559::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:56:05,569::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:56:05,569::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:56:05,569::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433765, 'last-update-local-ts': 1384433765.5692029, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:05,570::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:56:05,570::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433766, 'last-update-local-ts': 1384433765.5692029, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:05,570::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:56:15,662::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:56:15,954::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:56:16,056::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:56:16,128::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:56:16,128::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:56:16,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:56:16,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:16,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:56:16,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:56:16,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 14:56:16,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:16,132::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433776|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433776 (Thu Nov 14 14:56:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:56:16,132::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:56:16,144::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:56:16,144::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:56:16,144::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433776, 'last-update-local-ts': 1384433776.1441159, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:16,145::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:56:16,145::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433776, 'last-update-local-ts': 1384433776.1441159, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:16,145::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:56:26,219::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:56:26,473::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:56:26,554::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:56:26,615::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:56:26,615::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:56:26,616::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:56:26,616::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:26,616::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:56:26,617::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:56:26,617::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.715 >MainThread::INFO::2013-11-14 14:56:26,617::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:26,618::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433786|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433786 (Thu Nov 14 14:56:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.715 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:56:26,618::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:56:26,626::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:56:26,627::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:56:26,627::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433786, 'last-update-local-ts': 1384433786.6266501, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:26,627::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:56:26,627::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433787, 'last-update-local-ts': 1384433786.6266501, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:26,627::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:56:36,711::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:56:36,964::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:56:37,045::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:56:37,105::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:56:37,105::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:56:37,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:56:37,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:37,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:56:37,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:56:37,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.605 >MainThread::INFO::2013-11-14 14:56:37,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:37,108::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433797|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433797 (Thu Nov 14 14:56:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.605 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:56:37,108::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:56:37,117::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:56:37,117::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:56:37,117::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433797, 'last-update-local-ts': 1384433797.1172249, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:37,118::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:56:37,118::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433797, 'last-update-local-ts': 1384433797.1172249, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:37,118::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:56:47,210::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:56:47,559::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:56:47,643::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:56:47,704::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:56:47,704::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:56:47,705::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:56:47,705::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:47,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 14:56:47,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:56:47,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.51 >MainThread::INFO::2013-11-14 14:56:47,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:47,707::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433807|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433807 (Thu Nov 14 14:56:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.51 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:56:47,708::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:56:47,716::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:56:47,717::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:56:47,717::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433807, 'last-update-local-ts': 1384433807.716562, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:47,717::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:56:47,717::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433808, 'last-update-local-ts': 1384433807.716562, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:47,717::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:56:57,792::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:56:58,046::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:56:58,159::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:56:58,219::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:56:58,220::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:56:58,226::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:56:58,226::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:58,227::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:56:58,227::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:56:58,227::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.43 >MainThread::INFO::2013-11-14 14:56:58,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:56:58,228::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433818|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433818 (Thu Nov 14 14:56:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.43 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:56:58,228::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:56:58,238::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:56:58,238::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:56:58,238::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433818, 'last-update-local-ts': 1384433818.2382109, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:58,239::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:56:58,239::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433818, 'last-update-local-ts': 1384433818.2382109, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:56:58,239::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:57:08,316::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:57:08,572::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:57:08,658::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:57:08,720::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:57:08,720::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:57:08,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:57:08,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:08,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:57:08,722::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:57:08,722::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.365 >MainThread::INFO::2013-11-14 14:57:08,722::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:08,723::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433828|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433828 (Thu Nov 14 14:57:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.365 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:57:08,723::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:57:08,733::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:57:08,733::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:57:08,733::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433828, 'last-update-local-ts': 1384433828.732955, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:08,733::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:57:08,734::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433829, 'last-update-local-ts': 1384433828.732955, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:08,734::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:57:18,809::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:57:19,057::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:57:19,125::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:57:19,185::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:57:19,186::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:57:19,186::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:57:19,186::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:19,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:57:19,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:57:19,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 14:57:19,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:19,188::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433839|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433839 (Thu Nov 14 14:57:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:57:19,188::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:57:19,196::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:57:19,197::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:57:19,197::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433839, 'last-update-local-ts': 1384433839.1966529, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:19,197::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:57:19,197::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433839, 'last-update-local-ts': 1384433839.1966529, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:19,197::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:57:29,285::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:57:29,610::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:57:29,789::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:57:29,851::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:57:29,851::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:57:29,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:57:29,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:29,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 14:57:29,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:57:29,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.26 >MainThread::INFO::2013-11-14 14:57:29,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:29,854::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433849|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433849 (Thu Nov 14 14:57:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.26 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:57:29,854::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:57:29,862::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:57:29,863::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:57:29,863::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433849, 'last-update-local-ts': 1384433849.8624959, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:29,863::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:57:29,863::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433850, 'last-update-local-ts': 1384433849.8624959, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:29,863::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:57:39,943::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:57:40,193::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:57:40,294::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:57:40,353::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:57:40,354::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:57:40,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:57:40,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:40,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:57:40,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:57:40,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.735 >MainThread::INFO::2013-11-14 14:57:40,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:40,356::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433860|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433860 (Thu Nov 14 14:57:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.735 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:57:40,357::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:57:40,365::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:57:40,366::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:57:40,366::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433860, 'last-update-local-ts': 1384433860.365571, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:40,366::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:57:40,366::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433860, 'last-update-local-ts': 1384433860.365571, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:40,366::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:57:50,436::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:57:50,702::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:57:50,792::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:57:50,859::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:57:50,859::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:57:50,859::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:57:50,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:50,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:57:50,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:57:50,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.625 >MainThread::INFO::2013-11-14 14:57:50,861::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:57:50,861::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433870|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433870 (Thu Nov 14 14:57:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.625 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:57:50,861::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:57:50,868::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:57:50,868::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:57:50,869::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433870, 'last-update-local-ts': 1384433870.8683431, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:50,869::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:57:50,869::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433871, 'last-update-local-ts': 1384433870.8683431, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:57:50,869::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:58:00,944::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:58:01,253::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:58:01,341::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:58:01,400::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:58:01,401::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:58:01,401::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:58:01,401::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:01,402::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:58:01,402::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:58:01,402::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.525 >MainThread::INFO::2013-11-14 14:58:01,402::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:01,404::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433881|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433881 (Thu Nov 14 14:58:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.525 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:58:01,404::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:58:01,413::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:58:01,413::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:58:01,413::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433881, 'last-update-local-ts': 1384433881.4131391, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:01,413::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:58:01,414::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433881, 'last-update-local-ts': 1384433881.4131391, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:01,414::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:58:11,499::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:58:11,775::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:58:11,830::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:58:11,891::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:58:11,891::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:58:11,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:58:11,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:11,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:58:11,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:58:11,893::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.485 >MainThread::INFO::2013-11-14 14:58:11,893::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:11,893::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433891|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433891 (Thu Nov 14 14:58:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.485 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:58:11,894::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:58:11,900::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:58:11,901::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:58:11,901::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433891, 'last-update-local-ts': 1384433891.9007211, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:11,901::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:58:11,901::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433891, 'last-update-local-ts': 1384433891.9007211, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:11,902::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:58:21,988::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:58:22,240::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:58:22,328::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:58:22,390::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:58:22,391::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:58:22,397::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:58:22,397::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:22,398::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:58:22,398::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:58:22,398::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.41 >MainThread::INFO::2013-11-14 14:58:22,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:22,399::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433902|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433902 (Thu Nov 14 14:58:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.41 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:58:22,399::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:58:22,407::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:58:22,407::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:58:22,408::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433902, 'last-update-local-ts': 1384433902.4073629, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:22,408::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:58:22,408::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433902, 'last-update-local-ts': 1384433902.4073629, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:22,408::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:58:32,494::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:58:32,748::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:58:32,823::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:58:32,885::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:58:32,885::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:58:32,885::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:58:32,886::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:32,886::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:58:32,886::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:58:32,886::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.345 >MainThread::INFO::2013-11-14 14:58:32,887::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:32,888::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433912|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433912 (Thu Nov 14 14:58:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.345 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:58:32,888::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:58:32,894::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:58:32,895::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:58:32,895::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433912, 'last-update-local-ts': 1384433912.894779, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:32,895::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:58:32,895::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433912, 'last-update-local-ts': 1384433912.894779, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:32,896::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:58:42,974::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:58:43,221::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:58:43,302::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:58:43,363::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:58:43,363::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:58:43,363::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:58:43,364::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:43,364::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:58:43,364::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:58:43,364::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.29 >MainThread::INFO::2013-11-14 14:58:43,365::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:43,365::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433923|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433923 (Thu Nov 14 14:58:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.29 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:58:43,365::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:58:43,372::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:58:43,372::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:58:43,373::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433923, 'last-update-local-ts': 1384433923.37238, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:43,373::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:58:43,373::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433923, 'last-update-local-ts': 1384433923.37238, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:43,373::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:58:53,447::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:58:53,700::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:58:53,764::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:58:53,823::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:58:53,824::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:58:53,830::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:58:53,830::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:53,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:58:53,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:58:53,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.245 >MainThread::INFO::2013-11-14 14:58:53,832::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:58:53,832::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433933|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433933 (Thu Nov 14 14:58:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.245 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:58:53,832::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:58:53,842::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:58:53,842::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:58:53,843::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433933, 'last-update-local-ts': 1384433933.8424489, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:53,843::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:58:53,843::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433933, 'last-update-local-ts': 1384433933.8424489, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:58:53,843::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:59:03,937::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:59:04,229::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:59:04,337::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:59:04,415::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:59:04,415::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:59:04,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:59:04,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:04,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7065 >MainThread::INFO::2013-11-14 14:59:04,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:59:04,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 14:59:04,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:04,418::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433944|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433944 (Thu Nov 14 14:59:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7065 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:59:04,418::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:59:04,427::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:59:04,427::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:59:04,427::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433944, 'last-update-local-ts': 1384433944.427211, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:04,428::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:59:04,428::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433944, 'last-update-local-ts': 1384433944.427211, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:04,428::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:59:14,509::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:59:14,781::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:59:14,873::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:59:14,934::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:59:14,934::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:59:14,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:59:14,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:14,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:59:14,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:59:14,942::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 14:59:14,942::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:14,942::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433954|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433954 (Thu Nov 14 14:59:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:59:14,942::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:59:14,952::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:59:14,953::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:59:14,953::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433954, 'last-update-local-ts': 1384433954.9527459, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:14,953::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:59:14,953::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433954, 'last-update-local-ts': 1384433954.9527459, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:14,954::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:59:25,046::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:59:25,297::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:59:25,371::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:59:25,431::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:59:25,432::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:59:25,432::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:59:25,432::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:25,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:59:25,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:59:25,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.15 >MainThread::INFO::2013-11-14 14:59:25,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:25,435::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433965|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433965 (Thu Nov 14 14:59:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.15 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:59:25,435::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:59:25,444::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:59:25,444::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:59:25,444::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433965, 'last-update-local-ts': 1384433965.4440241, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:25,444::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:59:25,445::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433965, 'last-update-local-ts': 1384433965.4440241, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:25,445::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:59:35,515::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:59:35,771::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:59:35,853::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:59:35,926::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:59:35,926::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:59:35,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:59:35,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:35,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:59:35,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:59:35,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.345 >MainThread::INFO::2013-11-14 14:59:35,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:35,928::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433975|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433975 (Thu Nov 14 14:59:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.345 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:59:35,928::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:59:35,937::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:59:35,937::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:59:35,938::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433975, 'last-update-local-ts': 1384433975.9374959, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:35,938::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:59:35,938::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433975, 'last-update-local-ts': 1384433975.9374959, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:35,938::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:59:46,011::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:59:46,267::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:59:46,321::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:59:46,380::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:59:46,381::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:59:46,388::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:59:46,388::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:46,389::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:59:46,389::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:59:46,389::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.295 >MainThread::INFO::2013-11-14 14:59:46,389::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:46,390::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433986|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433986 (Thu Nov 14 14:59:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.295 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:59:46,390::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:59:46,405::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:59:46,405::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:59:46,405::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433986, 'last-update-local-ts': 1384433986.4049931, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:46,405::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:59:46,406::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433986, 'last-update-local-ts': 1384433986.4049931, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:46,406::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 14:59:56,500::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 14:59:56,751::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 14:59:56,826::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 14:59:56,885::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 14:59:56,886::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 14:59:56,886::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 14:59:56,886::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:56,887::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 14:59:56,887::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 14:59:56,887::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.35 >MainThread::INFO::2013-11-14 14:59:56,888::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 14:59:56,889::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384433996|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384433996 (Thu Nov 14 14:59:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.35 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 14:59:56,889::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 14:59:56,901::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 14:59:56,901::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 14:59:56,901::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384433996, 'last-update-local-ts': 1384433996.901258, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:56,902::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 14:59:56,902::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384433996, 'last-update-local-ts': 1384433996.901258, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 14:59:56,902::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:00:06,986::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:00:07,246::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:00:07,387::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:00:07,453::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:00:07,454::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:00:07,454::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:00:07,454::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:07,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:00:07,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:00:07,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.295 >MainThread::INFO::2013-11-14 15:00:07,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:07,457::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434007|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434007 (Thu Nov 14 15:00:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.295 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:00:07,457::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:00:07,466::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:00:07,467::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:00:07,467::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434007, 'last-update-local-ts': 1384434007.4666331, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:07,467::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:00:07,467::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434006, 'last-update-local-ts': 1384434007.4666331, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:07,467::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:00:17,546::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:00:17,799::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:00:17,896::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:00:17,956::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:00:17,957::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:00:17,957::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:00:17,957::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:17,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:00:17,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:00:17,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.25 >MainThread::INFO::2013-11-14 15:00:17,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:17,959::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434017|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434017 (Thu Nov 14 15:00:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.25 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:00:17,959::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:00:17,967::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:00:17,968::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:00:17,968::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434017, 'last-update-local-ts': 1384434017.96784, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:17,968::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:00:17,968::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434017, 'last-update-local-ts': 1384434017.96784, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:17,969::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:00:28,048::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:00:28,301::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:00:28,387::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:00:28,447::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:00:28,447::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:00:28,454::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:00:28,454::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:28,454::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:00:28,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:00:28,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 15:00:28,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:28,456::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434028|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434028 (Thu Nov 14 15:00:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:00:28,457::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:00:28,466::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:00:28,466::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:00:28,466::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434028, 'last-update-local-ts': 1384434028.4663119, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:28,467::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:00:28,467::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434027, 'last-update-local-ts': 1384434028.4663119, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:28,467::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:00:38,554::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:00:38,811::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:00:38,866::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:00:38,926::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:00:38,926::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:00:38,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:00:38,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:38,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:00:38,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:00:38,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 15:00:38,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:38,928::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434038|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434038 (Thu Nov 14 15:00:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:00:38,929::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:00:38,938::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:00:38,938::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:00:38,938::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434038, 'last-update-local-ts': 1384434038.9380951, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:38,938::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:00:38,939::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434038, 'last-update-local-ts': 1384434038.9380951, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:38,939::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:00:49,009::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:00:49,255::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:00:49,339::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:00:49,398::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:00:49,399::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:00:49,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:00:49,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:49,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:00:49,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:00:49,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.295 >MainThread::INFO::2013-11-14 15:00:49,401::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:49,401::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434049|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434049 (Thu Nov 14 15:00:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.295 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:00:49,401::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:00:49,419::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:00:49,420::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:00:49,420::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434049, 'last-update-local-ts': 1384434049.4197381, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:49,420::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:00:49,420::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434048, 'last-update-local-ts': 1384434049.4197381, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:49,421::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:00:59,505::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:00:59,790::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:00:59,884::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:00:59,944::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:00:59,944::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:00:59,945::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:00:59,945::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:59,945::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:00:59,946::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:00:59,946::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.25 >MainThread::INFO::2013-11-14 15:00:59,946::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:00:59,947::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434059|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434059 (Thu Nov 14 15:00:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.25 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:00:59,948::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:00:59,954::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:00:59,955::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:00:59,955::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434059, 'last-update-local-ts': 1384434059.954808, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:59,955::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:00:59,955::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434059, 'last-update-local-ts': 1384434059.954808, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:00:59,956::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:01:10,057::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:01:10,336::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:01:10,468::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:01:10,537::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:01:10,537::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:01:10,538::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0015243902439 >MainThread::INFO::2013-11-14 15:01:10,538::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:10,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:01:10,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:01:10,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 15:01:10,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:10,541::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434070|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434070 (Thu Nov 14 15:01:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.0015243902439 > >MainThread::INFO::2013-11-14 15:01:10,542::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:01:10,551::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:01:10,552::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:01:10,552::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434070, 'last-update-local-ts': 1384434070.551558, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:10,552::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:01:10,552::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434069, 'last-update-local-ts': 1384434070.551558, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:10,552::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:01:20,624::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:01:20,880::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:01:20,954::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:01:21,043::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:01:21,043::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:01:21,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:01:21,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:21,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:01:21,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:01:21,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.25 >MainThread::INFO::2013-11-14 15:01:21,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:21,046::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434081|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434081 (Thu Nov 14 15:01:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.25 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:01:21,046::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:01:21,053::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:01:21,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:01:21,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434081, 'last-update-local-ts': 1384434081.05389, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:21,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:01:21,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434080, 'last-update-local-ts': 1384434081.05389, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:21,055::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:01:31,139::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:01:31,392::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:01:31,491::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:01:31,551::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:01:31,552::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:01:31,552::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:01:31,552::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:31,553::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:01:31,553::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:01:31,553::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.325 >MainThread::INFO::2013-11-14 15:01:31,554::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:31,554::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434091|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434091 (Thu Nov 14 15:01:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.325 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:01:31,554::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:01:31,562::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:01:31,563::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:01:31,563::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434091, 'last-update-local-ts': 1384434091.5628819, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:31,563::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:01:31,563::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434090, 'last-update-local-ts': 1384434091.5628819, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:31,564::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:01:41,666::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:01:41,920::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:01:42,002::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:01:42,070::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:01:42,070::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:01:42,071::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:01:42,071::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:42,071::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:01:42,072::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:01:42,072::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 15:01:42,072::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:42,073::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434102|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434102 (Thu Nov 14 15:01:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:01:42,073::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:01:42,081::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:01:42,082::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:01:42,082::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434102, 'last-update-local-ts': 1384434102.0818, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:42,082::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:01:42,082::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434101, 'last-update-local-ts': 1384434102.0818, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:42,083::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:01:52,164::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:01:52,423::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:01:52,505::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:01:52,566::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:01:52,566::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:01:52,566::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:01:52,567::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:52,567::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:01:52,567::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:01:52,567::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 15:01:52,568::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:01:52,568::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434112|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434112 (Thu Nov 14 15:01:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.265 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:01:52,568::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:01:52,578::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:01:52,578::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:01:52,578::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434112, 'last-update-local-ts': 1384434112.5780201, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:52,578::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:01:52,579::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434111, 'last-update-local-ts': 1384434112.5780201, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:01:52,579::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:02:02,658::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:02:02,920::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:02:02,991::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:02:03,085::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:02:03,085::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:02:03,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:02:03,091::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:03,092::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:02:03,092::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:02:03,093::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 15:02:03,093::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:03,094::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434123|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434123 (Thu Nov 14 15:02:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:02:03,095::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:02:03,106::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:02:03,106::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:02:03,106::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434123, 'last-update-local-ts': 1384434123.1062529, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:03,107::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:02:03,107::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434121, 'last-update-local-ts': 1384434123.1062529, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:03,107::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:02:13,202::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:02:13,479::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:02:13,557::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:02:13,639::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:02:13,639::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:02:13,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:02:13,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:13,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:02:13,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:02:13,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.74 >MainThread::INFO::2013-11-14 15:02:13,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:13,642::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434133|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434133 (Thu Nov 14 15:02:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.74 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:02:13,642::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:02:13,652::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:02:13,652::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:02:13,652::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434133, 'last-update-local-ts': 1384434133.6521649, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:13,653::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:02:13,653::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434132, 'last-update-local-ts': 1384434133.6521649, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:13,653::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:02:23,728::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:02:23,976::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:02:24,044::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:02:24,117::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:02:24,117::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:02:24,117::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:02:24,118::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:24,118::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:02:24,118::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:02:24,119::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.625 >MainThread::INFO::2013-11-14 15:02:24,119::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:24,119::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434144|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434144 (Thu Nov 14 15:02:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.625 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:02:24,119::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:02:24,128::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:02:24,128::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:02:24,128::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434144, 'last-update-local-ts': 1384434144.1279581, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:24,128::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:02:24,128::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434142, 'last-update-local-ts': 1384434144.1279581, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:24,129::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:02:34,200::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:02:34,471::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:02:34,564::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:02:34,623::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:02:34,623::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:02:34,624::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:02:34,624::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:34,624::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:02:34,624::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:02:34,625::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.53 >MainThread::INFO::2013-11-14 15:02:34,625::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:34,626::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434154|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434154 (Thu Nov 14 15:02:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.53 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:02:34,626::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:02:34,637::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:02:34,638::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:02:34,638::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434154, 'last-update-local-ts': 1384434154.637743, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:34,638::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:02:34,638::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434153, 'last-update-local-ts': 1384434154.637743, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:34,639::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:02:44,733::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:02:44,985::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:02:45,065::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:02:45,132::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:02:45,132::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:02:45,133::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:02:45,133::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:45,133::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:02:45,134::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:02:45,134::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.45 >MainThread::INFO::2013-11-14 15:02:45,134::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:45,135::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434165|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434165 (Thu Nov 14 15:02:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.45 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:02:45,135::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:02:45,152::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:02:45,152::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:02:45,152::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434165, 'last-update-local-ts': 1384434165.152277, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:45,153::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:02:45,153::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434163, 'last-update-local-ts': 1384434165.152277, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:45,153::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:02:55,239::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:02:55,491::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:02:55,571::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:02:55,630::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:02:55,630::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:02:55,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:02:55,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:55,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:02:55,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:02:55,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.38 >MainThread::INFO::2013-11-14 15:02:55,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:02:55,639::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434175|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434175 (Thu Nov 14 15:02:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.38 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:02:55,639::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:02:55,649::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:02:55,649::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:02:55,649::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434175, 'last-update-local-ts': 1384434175.6491621, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:55,650::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:02:55,650::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434174, 'last-update-local-ts': 1384434175.6491621, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:02:55,650::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:03:05,729::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:03:05,983::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:03:06,062::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:03:06,123::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:03:06,123::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:03:06,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:03:06,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:06,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:03:06,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:03:06,125::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.615 >MainThread::INFO::2013-11-14 15:03:06,125::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:06,126::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434186|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434186 (Thu Nov 14 15:03:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.615 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:03:06,126::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:03:06,135::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:03:06,135::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:03:06,135::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434186, 'last-update-local-ts': 1384434186.1351931, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:06,136::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:03:06,136::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434184, 'last-update-local-ts': 1384434186.1351931, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:06,136::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:03:16,200::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:03:16,460::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:03:16,529::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:03:16,588::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:03:16,589::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:03:16,589::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:03:16,589::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:16,590::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:03:16,590::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:03:16,590::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.52 >MainThread::INFO::2013-11-14 15:03:16,590::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:16,591::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434196|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434196 (Thu Nov 14 15:03:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.52 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:03:16,591::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:03:16,600::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:03:16,600::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:03:16,600::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434196, 'last-update-local-ts': 1384434196.600239, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:16,601::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:03:16,601::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434195, 'last-update-local-ts': 1384434196.600239, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:16,601::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:03:26,689::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:03:26,942::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:03:27,027::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:03:27,087::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:03:27,087::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:03:27,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:03:27,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:27,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:03:27,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:03:27,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.37 >MainThread::INFO::2013-11-14 15:03:27,089::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:27,089::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434207|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434207 (Thu Nov 14 15:03:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.37 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:03:27,089::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:03:27,102::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:03:27,102::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:03:27,102::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434207, 'last-update-local-ts': 1384434207.1021841, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:27,103::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:03:27,103::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434205, 'last-update-local-ts': 1384434207.1021841, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:27,103::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:03:37,191::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:03:37,460::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:03:37,553::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:03:37,613::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:03:37,614::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:03:37,614::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:03:37,614::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:37,615::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:03:37,615::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:03:37,615::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.755 >MainThread::INFO::2013-11-14 15:03:37,615::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:37,617::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434217|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434217 (Thu Nov 14 15:03:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.755 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:03:37,617::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:03:37,627::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:03:37,627::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:03:37,627::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434217, 'last-update-local-ts': 1384434217.62707, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:37,627::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:03:37,628::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434216, 'last-update-local-ts': 1384434217.62707, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:37,628::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:03:47,704::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:03:47,959::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:03:48,040::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:03:48,113::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:03:48,113::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:03:48,113::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:03:48,114::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:48,114::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:03:48,114::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:03:48,115::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.64 >MainThread::INFO::2013-11-14 15:03:48,115::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:48,115::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434228|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434228 (Thu Nov 14 15:03:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.64 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:03:48,115::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:03:48,124::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:03:48,124::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:03:48,125::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434228, 'last-update-local-ts': 1384434228.124424, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:48,125::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:03:48,125::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434226, 'last-update-local-ts': 1384434228.124424, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:48,125::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:03:58,198::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:03:58,453::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:03:58,507::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:03:58,571::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:03:58,571::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:03:58,571::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:03:58,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:58,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:03:58,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:03:58,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.54 >MainThread::INFO::2013-11-14 15:03:58,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:03:58,573::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434238|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434238 (Thu Nov 14 15:03:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.54 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:03:58,573::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:03:58,594::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:03:58,595::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:03:58,595::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434238, 'last-update-local-ts': 1384434238.5947151, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:58,595::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:03:58,595::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434237, 'last-update-local-ts': 1384434238.5947151, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:03:58,596::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:04:08,681::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:04:08,931::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:04:09,024::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:04:09,084::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:04:09,084::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:04:09,085::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:04:09,085::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:09,085::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:04:09,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:04:09,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.46 >MainThread::INFO::2013-11-14 15:04:09,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:09,087::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434249|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434249 (Thu Nov 14 15:04:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.46 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:04:09,087::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:04:09,099::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:04:09,099::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:04:09,099::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434249, 'last-update-local-ts': 1384434249.0992291, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:09,100::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:04:09,100::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434247, 'last-update-local-ts': 1384434249.0992291, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:09,100::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:04:19,166::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:04:19,430::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:04:19,523::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:04:19,599::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:04:19,600::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:04:19,600::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:04:19,600::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:19,601::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:04:19,601::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:04:19,601::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.46 >MainThread::INFO::2013-11-14 15:04:19,602::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:19,602::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434259|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434259 (Thu Nov 14 15:04:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.46 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:04:19,602::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:04:19,615::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:04:19,616::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:04:19,616::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434259, 'last-update-local-ts': 1384434259.615742, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:19,616::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:04:19,616::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434258, 'last-update-local-ts': 1384434259.615742, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:19,617::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:04:29,712::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:04:29,989::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:04:30,077::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:04:30,138::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:04:30,138::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:04:30,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:04:30,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:30,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7060 >MainThread::INFO::2013-11-14 15:04:30,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:04:30,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.39 >MainThread::INFO::2013-11-14 15:04:30,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:30,141::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434270|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434270 (Thu Nov 14 15:04:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.39 >engine-health=vm-down >gateway=True >mem-free=7060 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:04:30,141::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:04:30,149::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:04:30,150::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:04:30,150::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434270, 'last-update-local-ts': 1384434270.1495919, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:30,150::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:04:30,150::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434269, 'last-update-local-ts': 1384434270.1495919, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:30,150::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:04:40,229::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:04:41,328::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:04:41,404::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:04:41,465::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:04:41,466::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:04:41,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:04:41,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:41,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:04:41,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:04:41,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.33 >MainThread::INFO::2013-11-14 15:04:41,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:41,469::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434281|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434281 (Thu Nov 14 15:04:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.33 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:04:41,469::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:04:41,475::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:04:41,476::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:04:41,476::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434281, 'last-update-local-ts': 1384434281.475817, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:41,476::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:04:41,476::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434279, 'last-update-local-ts': 1384434281.475817, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:41,477::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:04:51,547::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:04:51,801::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:04:51,865::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:04:51,925::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:04:51,925::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:04:51,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:04:51,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:51,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7062 >MainThread::INFO::2013-11-14 15:04:51,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:04:51,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.28 >MainThread::INFO::2013-11-14 15:04:51,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:04:51,927::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434291|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434291 (Thu Nov 14 15:04:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.28 >engine-health=vm-down >gateway=True >mem-free=7062 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:04:51,928::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:04:51,935::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:04:51,935::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:04:51,936::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434291, 'last-update-local-ts': 1384434291.9353809, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:51,936::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:04:51,936::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434289, 'last-update-local-ts': 1384434291.9353809, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:04:51,936::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:05:02,023::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:05:02,272::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:05:02,376::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:05:02,437::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:05:02,437::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:05:02,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:05:02,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:02,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:05:02,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:05:02,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 15:05:02,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:02,440::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434302|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434302 (Thu Nov 14 15:05:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:05:02,441::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:05:02,448::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:05:02,449::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:05:02,449::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434302, 'last-update-local-ts': 1384434302.44858, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:02,449::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:05:02,449::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434300, 'last-update-local-ts': 1384434302.44858, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:02,449::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:05:12,526::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:05:12,781::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:05:12,873::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:05:12,939::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:05:12,939::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:05:12,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:05:12,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:12,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:05:12,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:05:12,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.26 >MainThread::INFO::2013-11-14 15:05:12,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:12,942::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434312|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434312 (Thu Nov 14 15:05:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.26 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:05:12,942::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:05:12,949::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:05:12,949::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:05:12,949::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434312, 'last-update-local-ts': 1384434312.9491961, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:12,950::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:05:12,950::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434310, 'last-update-local-ts': 1384434312.9491961, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:12,950::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:05:23,031::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:05:23,597::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:05:23,685::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:05:23,747::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:05:23,748::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:05:23,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:05:23,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:23,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:05:23,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:05:23,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.22 >MainThread::INFO::2013-11-14 15:05:23,750::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:23,751::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434323|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434323 (Thu Nov 14 15:05:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.22 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:05:23,751::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:05:23,759::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:05:23,760::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:05:23,760::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434323, 'last-update-local-ts': 1384434323.759665, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:23,760::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:05:23,760::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434321, 'last-update-local-ts': 1384434323.759665, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:23,760::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:05:33,839::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:05:34,091::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:05:34,174::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:05:34,236::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:05:34,236::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:05:34,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:05:34,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:34,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:05:34,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:05:34,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 15:05:34,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:34,238::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434334|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434334 (Thu Nov 14 15:05:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:05:34,239::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:05:34,247::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:05:34,248::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:05:34,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434334, 'last-update-local-ts': 1384434334.247894, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:34,248::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:05:34,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434331, 'last-update-local-ts': 1384434334.247894, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:34,249::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:05:44,322::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:05:44,609::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:05:44,681::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:05:44,741::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:05:44,741::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:05:44,741::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:05:44,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:44,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:05:44,742::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:05:44,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 15:05:44,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:44,743::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434344|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434344 (Thu Nov 14 15:05:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:05:44,743::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:05:44,752::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:05:44,753::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:05:44,753::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434344, 'last-update-local-ts': 1384434344.752686, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:44,753::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:05:44,753::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434342, 'last-update-local-ts': 1384434344.752686, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:44,753::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:05:54,833::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:05:55,082::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:05:55,170::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:05:55,232::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:05:55,232::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:05:55,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:05:55,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:55,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:05:55,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:05:55,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 15:05:55,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:05:55,235::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434355|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434355 (Thu Nov 14 15:05:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:05:55,235::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:05:55,244::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:05:55,245::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:05:55,245::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434355, 'last-update-local-ts': 1384434355.244566, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:55,245::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:05:55,245::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434352, 'last-update-local-ts': 1384434355.244566, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:05:55,245::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:06:05,322::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:06:05,586::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:06:05,676::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:06:05,743::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:06:05,743::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:06:05,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:06:05,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:05,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:06:05,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:06:05,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 15:06:05,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:05,746::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434365|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434365 (Thu Nov 14 15:06:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:06:05,746::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:06:05,757::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:06:05,757::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:06:05,757::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434365, 'last-update-local-ts': 1384434365.757242, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:05,758::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:06:05,758::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434363, 'last-update-local-ts': 1384434365.757242, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:05,758::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:06:15,840::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:06:16,095::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:06:16,189::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:06:16,250::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:06:16,250::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:06:16,251::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:06:16,251::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:16,251::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:06:16,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:06:16,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 15:06:16,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:16,253::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434376|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434376 (Thu Nov 14 15:06:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:06:16,253::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:06:16,262::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:06:16,263::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:06:16,263::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434376, 'last-update-local-ts': 1384434376.2626159, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:16,263::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:06:16,263::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434373, 'last-update-local-ts': 1384434376.2626159, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:16,263::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:06:26,344::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:06:26,593::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:06:26,673::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:06:26,735::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:06:26,735::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:06:26,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:06:26,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:26,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:06:26,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:06:26,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 15:06:26,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:26,737::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434386|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434386 (Thu Nov 14 15:06:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:06:26,738::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:06:26,746::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:06:26,747::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:06:26,747::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434386, 'last-update-local-ts': 1384434386.7465639, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:26,747::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:06:26,747::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434384, 'last-update-local-ts': 1384434386.7465639, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:26,747::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:06:36,819::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:06:37,076::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:06:37,143::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:06:37,203::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:06:37,203::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:06:37,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:06:37,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:37,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:06:37,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:06:37,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 15:06:37,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:37,212::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434397|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434397 (Thu Nov 14 15:06:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:06:37,212::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:06:37,224::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:06:37,225::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:06:37,225::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434397, 'last-update-local-ts': 1384434397.2245419, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:37,225::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:06:37,225::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434394, 'last-update-local-ts': 1384434397.2245419, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:37,225::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:06:47,311::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:06:47,558::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:06:47,661::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:06:47,721::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:06:47,722::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:06:47,722::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:06:47,722::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:47,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:06:47,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:06:47,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 15:06:47,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:47,725::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434407|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434407 (Thu Nov 14 15:06:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:06:47,725::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:06:47,733::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:06:47,734::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:06:47,734::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434407, 'last-update-local-ts': 1384434407.7336619, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:47,734::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:06:47,734::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434404, 'last-update-local-ts': 1384434407.7336619, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:47,734::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:06:57,812::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:06:58,073::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:06:58,171::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:06:58,231::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:06:58,232::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:06:58,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:06:58,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:58,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:06:58,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:06:58,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 15:06:58,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:06:58,234::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434418|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434418 (Thu Nov 14 15:06:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:06:58,234::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:06:58,243::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:06:58,243::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:06:58,243::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434418, 'last-update-local-ts': 1384434418.2430611, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:58,243::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:06:58,244::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434415, 'last-update-local-ts': 1384434418.2430611, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:06:58,244::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:07:08,330::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:07:08,582::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:07:08,675::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:07:08,735::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:07:08,736::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:07:08,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:07:08,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:08,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:07:08,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:07:08,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 15:07:08,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:08,739::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434428|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434428 (Thu Nov 14 15:07:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:07:08,739::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:07:08,748::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:07:08,748::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:07:08,748::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434428, 'last-update-local-ts': 1384434428.7481251, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:08,748::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:07:08,749::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434425, 'last-update-local-ts': 1384434428.7481251, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:08,749::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:07:18,814::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:07:19,067::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:07:19,132::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:07:19,191::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:07:19,192::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:07:19,192::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:07:19,192::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:19,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:07:19,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:07:19,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 15:07:19,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:19,194::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434439|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434439 (Thu Nov 14 15:07:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:07:19,194::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:07:19,205::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:07:19,205::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:07:19,206::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434439, 'last-update-local-ts': 1384434439.205493, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:19,206::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:07:19,206::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434436, 'last-update-local-ts': 1384434439.205493, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:19,206::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:07:29,287::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:07:29,552::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:07:29,675::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:07:29,741::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:07:29,742::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:07:29,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:07:29,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:29,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:07:29,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:07:29,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 15:07:29,750::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:29,750::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434449|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434449 (Thu Nov 14 15:07:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:07:29,750::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:07:29,764::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:07:29,765::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:07:29,765::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434449, 'last-update-local-ts': 1384434449.764715, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:29,765::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:07:29,765::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434446, 'last-update-local-ts': 1384434449.764715, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:29,766::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:07:39,853::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:07:40,162::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:07:40,259::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:07:40,320::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:07:40,320::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:07:40,321::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:07:40,321::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:40,321::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:07:40,321::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:07:40,322::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 15:07:40,322::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:40,323::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434460|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434460 (Thu Nov 14 15:07:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:07:40,323::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:07:40,330::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:07:40,330::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:07:40,331::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434460, 'last-update-local-ts': 1384434460.3303771, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:40,331::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:07:40,331::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434457, 'last-update-local-ts': 1384434460.3303771, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:40,331::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:07:50,408::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:07:50,666::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:07:50,756::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:07:50,817::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:07:50,817::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:07:50,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:07:50,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:50,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:07:50,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:07:50,819::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 15:07:50,819::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:07:50,819::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434470|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434470 (Thu Nov 14 15:07:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:07:50,820::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:07:50,828::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:07:50,828::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:07:50,829::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434470, 'last-update-local-ts': 1384434470.828377, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:50,829::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:07:50,829::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434467, 'last-update-local-ts': 1384434470.828377, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:07:50,829::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:08:00,905::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:08:01,161::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:08:01,251::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:08:01,316::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:08:01,316::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:08:01,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:08:01,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:01,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:08:01,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:08:01,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 15:08:01,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:01,319::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434481|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434481 (Thu Nov 14 15:08:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:08:01,320::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:08:01,328::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:08:01,328::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:08:01,328::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434481, 'last-update-local-ts': 1384434481.328186, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:01,329::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:08:01,329::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434478, 'last-update-local-ts': 1384434481.328186, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:01,329::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:08:11,407::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:08:11,663::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:08:11,739::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:08:11,806::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:08:11,807::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:08:11,807::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:08:11,807::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:11,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:08:11,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:08:11,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 15:08:11,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:11,809::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434491|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434491 (Thu Nov 14 15:08:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:08:11,809::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:08:11,829::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:08:11,829::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:08:11,830::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434491, 'last-update-local-ts': 1384434491.829437, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:11,830::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:08:11,830::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434488, 'last-update-local-ts': 1384434491.829437, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:11,830::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:08:21,907::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:08:22,156::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:08:22,214::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:08:22,275::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:08:22,275::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:08:22,282::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:08:22,282::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:22,282::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:08:22,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:08:22,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 15:08:22,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:22,284::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434502|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434502 (Thu Nov 14 15:08:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:08:22,284::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:08:22,293::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:08:22,293::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:08:22,294::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434502, 'last-update-local-ts': 1384434502.293412, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:22,294::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:08:22,294::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434499, 'last-update-local-ts': 1384434502.293412, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:22,294::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:08:32,393::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:08:32,646::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:08:32,772::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:08:32,842::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:08:32,842::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:08:32,842::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:08:32,843::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:32,843::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:08:32,843::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:08:32,843::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 15:08:32,844::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:32,845::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434512|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434512 (Thu Nov 14 15:08:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:08:32,845::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:08:32,854::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:08:32,854::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:08:32,854::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434512, 'last-update-local-ts': 1384434512.8540931, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:32,854::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:08:32,855::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434509, 'last-update-local-ts': 1384434512.8540931, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:32,855::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:08:42,939::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:08:43,198::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:08:43,291::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:08:43,352::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:08:43,352::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:08:43,353::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:08:43,353::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:43,353::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:08:43,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:08:43,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 15:08:43,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:43,355::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434523|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434523 (Thu Nov 14 15:08:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:08:43,355::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:08:43,364::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:08:43,364::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:08:43,364::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434523, 'last-update-local-ts': 1384434523.364208, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:43,365::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:08:43,365::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434519, 'last-update-local-ts': 1384434523.364208, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:43,365::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:08:53,460::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:08:53,715::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:08:53,794::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:08:53,861::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:08:53,862::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:08:53,862::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:08:53,862::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:53,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:08:53,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:08:53,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 15:08:53,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:08:53,864::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434533|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434533 (Thu Nov 14 15:08:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:08:53,865::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:08:53,872::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:08:53,872::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:08:53,872::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434533, 'last-update-local-ts': 1384434533.872015, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:53,872::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:08:53,873::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434530, 'last-update-local-ts': 1384434533.872015, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:08:53,873::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:09:03,945::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:09:04,206::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:09:04,276::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:09:04,337::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:09:04,337::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:09:04,344::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:09:04,344::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:04,344::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:09:04,345::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:09:04,345::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 15:09:04,345::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:04,346::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434544|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434544 (Thu Nov 14 15:09:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:09:04,346::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:09:04,356::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:09:04,356::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:09:04,356::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434544, 'last-update-local-ts': 1384434544.3562961, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:04,357::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:09:04,357::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434540, 'last-update-local-ts': 1384434544.3562961, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:04,357::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:09:14,438::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:09:14,707::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:09:14,800::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:09:14,863::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:09:14,863::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:09:14,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:09:14,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:14,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:09:14,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:09:14,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 15:09:14,865::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:14,866::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434554|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434554 (Thu Nov 14 15:09:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:09:14,866::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:09:14,875::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:09:14,875::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:09:14,875::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434554, 'last-update-local-ts': 1384434554.8751719, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:14,876::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:09:14,876::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434551, 'last-update-local-ts': 1384434554.8751719, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:14,876::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:09:24,953::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:09:25,202::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:09:25,281::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:09:25,349::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:09:25,349::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:09:25,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:09:25,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:25,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:09:25,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:09:25,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 15:09:25,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:25,352::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434565|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434565 (Thu Nov 14 15:09:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:09:25,352::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:09:25,360::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:09:25,360::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:09:25,360::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434565, 'last-update-local-ts': 1384434565.360091, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:25,360::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:09:25,361::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434561, 'last-update-local-ts': 1384434565.360091, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:25,361::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:09:35,441::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:09:35,695::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:09:35,785::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:09:35,845::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:09:35,845::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:09:35,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:09:35,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:35,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:09:35,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:09:35,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 15:09:35,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:35,854::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434575|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434575 (Thu Nov 14 15:09:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:09:35,854::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:09:35,863::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:09:35,864::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:09:35,864::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434575, 'last-update-local-ts': 1384434575.863858, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:35,864::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:09:35,864::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434572, 'last-update-local-ts': 1384434575.863858, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:35,865::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:09:45,954::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:09:46,206::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:09:46,275::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:09:46,368::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:09:46,369::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:09:46,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:09:46,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:46,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:09:46,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:09:46,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 15:09:46,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:46,371::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434586|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434586 (Thu Nov 14 15:09:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:09:46,372::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:09:46,381::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:09:46,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:09:46,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434586, 'last-update-local-ts': 1384434586.3810019, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:46,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:09:46,382::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434582, 'last-update-local-ts': 1384434586.3810019, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:46,382::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:09:56,458::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:09:56,726::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:09:56,781::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:09:56,847::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:09:56,847::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:09:56,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:09:56,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:56,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:09:56,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:09:56,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.54 >MainThread::INFO::2013-11-14 15:09:56,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:09:56,849::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434596|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434596 (Thu Nov 14 15:09:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.54 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:09:56,850::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:09:56,858::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:09:56,858::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:09:56,859::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434596, 'last-update-local-ts': 1384434596.858505, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:56,859::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:09:56,859::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434593, 'last-update-local-ts': 1384434596.858505, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:09:56,859::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:10:06,964::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:10:07,230::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:10:07,367::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:10:07,438::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:10:07,438::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:10:07,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:10:07,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:07,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:10:07,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:10:07,440::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.385 >MainThread::INFO::2013-11-14 15:10:07,440::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:07,441::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434607|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434607 (Thu Nov 14 15:10:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.385 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:10:07,442::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:10:07,452::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:10:07,453::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:10:07,453::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434607, 'last-update-local-ts': 1384434607.452852, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:07,453::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:10:07,453::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434603, 'last-update-local-ts': 1384434607.452852, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:07,454::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:10:17,533::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:10:17,787::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:10:17,883::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:10:17,944::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:10:17,944::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:10:17,951::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:10:17,951::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:17,951::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:10:17,952::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:10:17,952::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.325 >MainThread::INFO::2013-11-14 15:10:17,952::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:17,953::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434617|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434617 (Thu Nov 14 15:10:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.325 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:10:17,953::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:10:17,963::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:10:17,964::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:10:17,964::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434617, 'last-update-local-ts': 1384434617.9636879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:17,964::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:10:17,964::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434613, 'last-update-local-ts': 1384434617.9636879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:17,965::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:10:28,061::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:10:28,312::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:10:28,392::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:10:28,460::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:10:28,460::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:10:28,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:10:28,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:28,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:10:28,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:10:28,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.425 >MainThread::INFO::2013-11-14 15:10:28,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:28,463::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434628|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434628 (Thu Nov 14 15:10:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.425 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:10:28,463::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:10:28,474::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:10:28,474::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:10:28,475::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434628, 'last-update-local-ts': 1384434628.4744339, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:28,475::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:10:28,475::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434624, 'last-update-local-ts': 1384434628.4744339, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:28,475::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:10:38,561::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:10:38,818::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:10:38,904::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:10:38,967::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:10:38,967::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:10:38,974::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:10:38,974::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:38,974::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:10:38,975::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:10:38,975::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.36 >MainThread::INFO::2013-11-14 15:10:38,975::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:38,976::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434638|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434638 (Thu Nov 14 15:10:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.36 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:10:38,976::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:10:38,986::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:10:38,986::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:10:38,986::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434638, 'last-update-local-ts': 1384434638.986253, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:38,987::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:10:38,987::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434634, 'last-update-local-ts': 1384434638.986253, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:38,987::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:10:49,068::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:10:49,317::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:10:49,370::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:10:49,429::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:10:49,429::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:10:49,430::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:10:49,430::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:49,430::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:10:49,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:10:49,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.305 >MainThread::INFO::2013-11-14 15:10:49,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:49,432::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434649|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434649 (Thu Nov 14 15:10:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.305 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:10:49,432::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:10:49,442::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:10:49,442::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:10:49,442::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434649, 'last-update-local-ts': 1384434649.4418919, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:49,442::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:10:49,442::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434645, 'last-update-local-ts': 1384434649.4418919, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:10:49,443::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:10:59,540::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:10:59,810::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:10:59,909::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:10:59,980::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:10:59,980::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:10:59,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:10:59,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:59,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7060 >MainThread::INFO::2013-11-14 15:10:59,984::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:10:59,990::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 15:10:59,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:10:59,991::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434659|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434659 (Thu Nov 14 15:10:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7060 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:10:59,991::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:11:00,001::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:11:00,001::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:11:00,001::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434659, 'last-update-local-ts': 1384434660.0010481, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:00,001::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:11:00,002::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434655, 'last-update-local-ts': 1384434660.0010481, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:00,002::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:11:10,085::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:11:10,356::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:11:10,443::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:11:10,503::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:11:10,503::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:11:10,510::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:11:10,510::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:10,510::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:11:10,511::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:11:10,511::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 15:11:10,511::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:10,512::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434670|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434670 (Thu Nov 14 15:11:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:11:10,512::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:11:10,519::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:11:10,520::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:11:10,520::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434670, 'last-update-local-ts': 1384434670.519768, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:10,520::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:11:10,521::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434666, 'last-update-local-ts': 1384434670.519768, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:10,521::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:11:20,596::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:11:20,851::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:11:20,945::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:11:21,010::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:11:21,010::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:11:21,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:11:21,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:21,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7064 >MainThread::INFO::2013-11-14 15:11:21,012::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:11:21,012::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 15:11:21,012::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:21,013::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434681|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434681 (Thu Nov 14 15:11:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7064 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:11:21,014::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:11:21,022::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:11:21,023::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:11:21,023::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434681, 'last-update-local-ts': 1384434681.0226331, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:21,023::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:11:21,023::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434676, 'last-update-local-ts': 1384434681.0226331, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:21,023::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:11:31,121::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:11:31,373::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:11:31,474::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:11:31,535::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:11:31,535::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:11:31,536::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:11:31,536::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:31,536::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:11:31,536::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:11:31,537::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 15:11:31,537::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:31,537::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434691|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434691 (Thu Nov 14 15:11:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:11:31,538::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:11:31,545::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:11:31,545::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:11:31,546::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434691, 'last-update-local-ts': 1384434691.545315, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:31,546::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:11:31,546::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434687, 'last-update-local-ts': 1384434691.545315, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:31,546::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:11:41,632::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:11:41,885::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:11:41,987::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:11:42,048::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:11:42,049::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:11:42,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:11:42,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:42,050::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:11:42,050::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:11:42,050::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.165 >MainThread::INFO::2013-11-14 15:11:42,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:42,052::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434702|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434702 (Thu Nov 14 15:11:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.165 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:11:42,052::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:11:42,060::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:11:42,061::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:11:42,061::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434702, 'last-update-local-ts': 1384434702.0606899, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:42,061::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:11:42,061::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434697, 'last-update-local-ts': 1384434702.0606899, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:42,062::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:11:52,138::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:11:52,399::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:11:52,479::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:11:52,540::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:11:52,540::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:11:52,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:11:52,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:52,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7062 >MainThread::INFO::2013-11-14 15:11:52,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:11:52,542::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 15:11:52,542::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:11:52,542::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434712|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434712 (Thu Nov 14 15:11:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7062 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:11:52,543::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:11:52,549::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:11:52,549::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:11:52,550::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434712, 'last-update-local-ts': 1384434712.5495329, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:52,550::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:11:52,550::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434708, 'last-update-local-ts': 1384434712.5495329, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:11:52,550::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:12:02,615::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:12:02,880::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:12:02,941::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:12:03,000::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:12:03,000::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:12:03,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:12:03,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:03,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:12:03,002::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:12:03,002::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:12:03,002::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:03,003::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434723|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434723 (Thu Nov 14 15:12:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:12:03,003::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:12:03,012::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:12:03,012::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:12:03,012::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434723, 'last-update-local-ts': 1384434723.012049, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:03,012::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:12:03,013::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434718, 'last-update-local-ts': 1384434723.012049, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:03,013::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:12:13,113::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:12:13,362::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:12:13,446::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:12:13,527::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:12:13,528::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:12:13,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:12:13,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:13,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:12:13,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:12:13,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 15:12:13,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:13,531::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434733|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434733 (Thu Nov 14 15:12:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:12:13,531::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:12:13,540::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:12:13,540::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:12:13,540::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434733, 'last-update-local-ts': 1384434733.5400391, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:13,540::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:12:13,541::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434729, 'last-update-local-ts': 1384434733.5400391, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:13,541::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:12:23,615::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:12:23,937::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:12:24,033::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:12:24,094::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:12:24,094::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:12:24,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:12:24,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:24,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7063 >MainThread::INFO::2013-11-14 15:12:24,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:12:24,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:12:24,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:24,103::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434744|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434744 (Thu Nov 14 15:12:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7063 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:12:24,103::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:12:24,111::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:12:24,112::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:12:24,112::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434744, 'last-update-local-ts': 1384434744.111654, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:24,112::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:12:24,112::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434739, 'last-update-local-ts': 1384434744.111654, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:24,112::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:12:34,216::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:12:34,473::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:12:34,571::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:12:34,639::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:12:34,639::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:12:34,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:12:34,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:34,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7062 >MainThread::INFO::2013-11-14 15:12:34,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:12:34,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 15:12:34,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:34,642::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434754|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434754 (Thu Nov 14 15:12:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7062 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:12:34,642::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:12:34,651::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:12:34,652::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:12:34,652::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434754, 'last-update-local-ts': 1384434754.6515729, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:34,652::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:12:34,652::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434751, 'last-update-local-ts': 1384434754.6515729, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:34,652::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:12:44,723::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:12:51,000::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:12:51,195::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:12:51,258::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:12:51,258::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:12:51,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:12:51,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:51,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7051 >MainThread::INFO::2013-11-14 15:12:51,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:12:51,260::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 15:12:51,260::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:12:51,261::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434771|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434771 (Thu Nov 14 15:12:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7051 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:12:51,261::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:12:51,268::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:12:51,268::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:12:51,268::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434771, 'last-update-local-ts': 1384434771.268259, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:51,269::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:12:51,269::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434772, 'last-update-local-ts': 1384434771.268259, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:12:51,269::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:13:01,340::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:13:01,598::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:13:01,696::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:13:01,762::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:13:01,762::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:13:01,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:13:01,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:01,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7061 >MainThread::INFO::2013-11-14 15:13:01,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:13:01,764::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 15:13:01,764::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:01,764::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434781|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434781 (Thu Nov 14 15:13:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7061 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:13:01,765::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:13:01,773::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:13:01,773::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:13:01,773::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434781, 'last-update-local-ts': 1384434781.7733281, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:13:01,774::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:13:01,774::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434783, 'last-update-local-ts': 1384434781.7733281, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:13:01,774::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:13:11,848::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:13:12,100::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:13:12,165::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:13:12,223::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:13:12,223::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:13:12,224::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:13:12,224::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:12,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7059 >MainThread::INFO::2013-11-14 15:13:12,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:13:12,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 15:13:12,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:12,227::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434792|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434792 (Thu Nov 14 15:13:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7059 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:13:12,227::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:13:12,235::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:13:12,236::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:13:12,236::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434792, 'last-update-local-ts': 1384434792.2356801, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:13:12,236::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:13:12,236::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434794, 'last-update-local-ts': 1384434792.2356801, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 15:13:12,236::hosted_engine::876::HostedEngine::(_handle_off) Engine down and local host has best score (2400), attempting to start engine VM >MainThread::INFO::2013-11-14 15:13:12,237::hosted_engine::932::HostedEngine::(_clean_vdsm_state) Ensuring VDSM state is clear for engine VM >MainThread::INFO::2013-11-14 15:13:12,291::hosted_engine::944::HostedEngine::(_clean_vdsm_state) Vdsm state for VM clean >MainThread::INFO::2013-11-14 15:13:12,291::hosted_engine::903::HostedEngine::(_start_engine_vm) Starting vm using `/usr/sbin/hosted-engine --vm-start` >MainThread::INFO::2013-11-14 15:13:12,549::hosted_engine::908::HostedEngine::(_start_engine_vm) stdout: >e4697eab-3d16-4ce6-9396-804fdbe53291 > Status = WaitForLaunch > nicModel = rtl8139,pv > emulatedMachine = rhel6.5.0 > pid = 0 > displayIp = 0 > devices = [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': '17b10cd5-13fa-4f41-b78f-10635443cfd1', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'type': 'disk', 'format': 'raw', 'bootOrder': '1', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'volumeID': '7e8c4fa6-75ab-43d2-a9e3-8644d24aaa7d', 'imageID': 'b7c0de86-7b2c-43ea-afc6-e79360de9d95', 'specParams': {}, 'readonly': 'false', 'domainID': '354a5aac-b271-49fe-bb53-7064bb1fbec2', 'deviceId': 'b7c0de86-7b2c-43ea-afc6-e79360de9d95', 'poolID': '00000000-0000-0000-0000-000000000000', 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'optional': 'false'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:17:fb:01', 'linkActive': 'true', 'network': 'rhevm', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': '33fd2981-3248-441a-9ccd-c264b45910be', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '2436cb00-2e37-4f4f-9c91-1fc6dc613abc', 'alias': 'console0'}] > smp = 2 > vmType = kvm > display = vnc > displaySecurePort = -1 > memSize = 4096 > displayPort = -1 > cpuType = Conroe > spiceSecureChannels = smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir > vmName = HostedEngine > clientIp = > >MainThread::INFO::2013-11-14 15:13:12,549::hosted_engine::909::HostedEngine::(_start_engine_vm) stderr: >MainThread::ERROR::2013-11-14 15:13:12,550::hosted_engine::922::HostedEngine::(_start_engine_vm) Engine VM started on localhost >MainThread::INFO::2013-11-14 15:13:22,675::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:13:22,952::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:13:23,302::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:13:23,398::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:13:23,399::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:13:23,405::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:13:23,406::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:23,406::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7012 >MainThread::INFO::2013-11-14 15:13:23,407::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:13:23,407::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 15:13:23,407::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:23,408::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434803|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434803 (Thu Nov 14 15:13:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-up bad-health-status >gateway=True >mem-free=7012 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:13:23,408::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:13:23,419::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:13:23,420::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:13:23,420::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434803, 'last-update-local-ts': 1384434803.4196589, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:13:23,420::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:13:23,420::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434804, 'last-update-local-ts': 1384434803.4196589, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 15:13:23,421::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 300 seconds >MainThread::INFO::2013-11-14 15:13:33,537::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:13:33,798::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:13:33,934::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:13:34,045::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:13:34,046::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:13:34,046::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0015243902439 >MainThread::INFO::2013-11-14 15:13:34,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:34,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6994 >MainThread::INFO::2013-11-14 15:13:34,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:13:34,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 15:13:34,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:34,049::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434814|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434814 (Thu Nov 14 15:13:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6994 >mem-load=0.0015243902439 > >MainThread::INFO::2013-11-14 15:13:34,049::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:13:34,077::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:13:34,077::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:13:34,078::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434814, 'last-update-local-ts': 1384434814.0773971, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:13:34,078::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:13:34,078::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434814, 'last-update-local-ts': 1384434814.0773971, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 15:13:34,078::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 289 seconds >MainThread::INFO::2013-11-14 15:13:44,242::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:13:44,541::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:13:44,699::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:13:44,761::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:13:44,761::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:13:44,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00215955284553 >MainThread::INFO::2013-11-14 15:13:44,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:44,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6533 >MainThread::INFO::2013-11-14 15:13:44,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:13:44,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.41 >MainThread::INFO::2013-11-14 15:13:44,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:44,764::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434824|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434824 (Thu Nov 14 15:13:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.41 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6533 >mem-load=0.00215955284553 > >MainThread::INFO::2013-11-14 15:13:44,764::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:13:44,780::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:13:44,780::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:13:44,780::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434824, 'last-update-local-ts': 1384434824.779916, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:13:44,780::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:13:44,781::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434825, 'last-update-local-ts': 1384434824.779916, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 15:13:44,781::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 279 seconds >MainThread::INFO::2013-11-14 15:13:54,901::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:13:55,162::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:13:55,242::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:13:55,309::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:13:55,309::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:13:55,310::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00228658536585 >MainThread::INFO::2013-11-14 15:13:55,310::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:55,310::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6525 >MainThread::INFO::2013-11-14 15:13:55,311::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:13:55,311::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.345 >MainThread::INFO::2013-11-14 15:13:55,311::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:13:55,312::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434835|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434835 (Thu Nov 14 15:13:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.345 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6525 >mem-load=0.00228658536585 > >MainThread::INFO::2013-11-14 15:13:55,312::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:13:55,321::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:13:55,321::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:13:55,322::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434835, 'last-update-local-ts': 1384434835.3214059, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:13:55,322::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:13:55,322::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434835, 'last-update-local-ts': 1384434835.3214059, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 15:13:55,322::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 268 seconds >MainThread::INFO::2013-11-14 15:14:05,454::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:14:05,749::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:14:05,883::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:14:05,946::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:14:05,946::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:14:05,954::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00228658536585 >MainThread::INFO::2013-11-14 15:14:05,955::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:05,955::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6473 >MainThread::INFO::2013-11-14 15:14:05,956::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:14:05,956::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 15:14:05,957::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:05,957::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434845|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434845 (Thu Nov 14 15:14:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6473 >mem-load=0.00228658536585 > >MainThread::INFO::2013-11-14 15:14:05,958::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:14:05,966::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:14:05,967::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:14:05,967::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434845, 'last-update-local-ts': 1384434845.966754, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:05,967::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:14:05,967::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434846, 'last-update-local-ts': 1384434845.966754, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 15:14:05,968::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 258 seconds >MainThread::INFO::2013-11-14 15:14:16,110::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:14:16,408::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:14:16,604::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:14:16,669::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:14:16,669::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:14:16,670::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0025406504065 >MainThread::INFO::2013-11-14 15:14:16,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:16,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6313 >MainThread::INFO::2013-11-14 15:14:16,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:14:16,672::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.36 >MainThread::INFO::2013-11-14 15:14:16,672::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:16,673::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434856|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434856 (Thu Nov 14 15:14:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.36 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6313 >mem-load=0.0025406504065 > >MainThread::INFO::2013-11-14 15:14:16,673::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:14:16,704::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:14:16,705::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:14:16,705::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434856, 'last-update-local-ts': 1384434856.704566, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:16,705::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:14:16,705::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434856, 'last-update-local-ts': 1384434856.704566, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 15:14:16,706::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 247 seconds >MainThread::INFO::2013-11-14 15:14:26,864::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:14:27,196::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:14:27,371::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:14:27,447::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:14:27,448::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:14:27,457::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0030487804878 >MainThread::INFO::2013-11-14 15:14:27,457::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:27,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 5946 >MainThread::INFO::2013-11-14 15:14:27,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:14:27,464::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.03 >MainThread::INFO::2013-11-14 15:14:27,465::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:27,465::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434867|1|2300|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434867 (Thu Nov 14 15:14:27 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=1.03 >engine-health=vm-up bad-health-status >gateway=True >mem-free=5946 >mem-load=0.0030487804878 > >MainThread::INFO::2013-11-14 15:14:27,465::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:14:27,487::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:14:27,488::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:14:27,488::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434867, 'last-update-local-ts': 1384434867.4877529, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:27,488::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:14:27,489::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434867, 'last-update-local-ts': 1384434867.4877529, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 15:14:27,489::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 236 seconds >MainThread::INFO::2013-11-14 15:14:37,560::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:14:37,833::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:14:37,898::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:14:37,965::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:14:37,965::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:14:37,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00368394308943 >MainThread::INFO::2013-11-14 15:14:37,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:37,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 5594 >MainThread::INFO::2013-11-14 15:14:37,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:14:37,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.95 >MainThread::INFO::2013-11-14 15:14:37,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:37,968::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434877|1|2300|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434877 (Thu Nov 14 15:14:37 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.95 >engine-health=vm-up good-health-status >gateway=True >mem-free=5594 >mem-load=0.00368394308943 > >MainThread::INFO::2013-11-14 15:14:37,969::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:14:37,976::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:14:37,977::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:14:37,977::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434877, 'last-update-local-ts': 1384434877.976594, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:37,977::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:14:37,977::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434877, 'last-update-local-ts': 1384434877.976594, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:37,977::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:14:48,077::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:14:48,334::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:14:48,418::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:14:48,484::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:14:48,485::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:14:48,485::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00381097560976 >MainThread::INFO::2013-11-14 15:14:48,485::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:48,486::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 5500 >MainThread::INFO::2013-11-14 15:14:48,486::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:14:48,486::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.955 >MainThread::INFO::2013-11-14 15:14:48,487::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:48,487::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434888|1|2300|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434888 (Thu Nov 14 15:14:48 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.955 >engine-health=vm-up good-health-status >gateway=True >mem-free=5500 >mem-load=0.00381097560976 > >MainThread::INFO::2013-11-14 15:14:48,487::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:14:48,496::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:14:48,496::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:14:48,496::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434888, 'last-update-local-ts': 1384434888.4959271, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:48,496::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:14:48,496::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434887, 'last-update-local-ts': 1384434888.4959271, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:48,497::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:14:58,638::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:14:58,912::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:14:59,002::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:14:59,063::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:14:59,063::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:14:59,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00419207317073 >MainThread::INFO::2013-11-14 15:14:59,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:59,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 5309 >MainThread::INFO::2013-11-14 15:14:59,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:14:59,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.965 >MainThread::INFO::2013-11-14 15:14:59,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:14:59,066::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434899|1|2300|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434899 (Thu Nov 14 15:14:59 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.965 >engine-health=vm-up good-health-status >gateway=True >mem-free=5309 >mem-load=0.00419207317073 > >MainThread::INFO::2013-11-14 15:14:59,066::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:14:59,080::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:14:59,081::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:14:59,081::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434899, 'last-update-local-ts': 1384434899.080704, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:59,081::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:14:59,081::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434898, 'last-update-local-ts': 1384434899.080704, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:14:59,082::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:15:09,167::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:15:09,420::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:15:09,530::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:15:09,591::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:15:09,591::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:15:09,591::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00431910569106 >MainThread::INFO::2013-11-14 15:15:09,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:09,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 5197 >MainThread::INFO::2013-11-14 15:15:09,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:15:09,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.9 >MainThread::INFO::2013-11-14 15:15:09,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:09,593::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434909|1|2300|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434909 (Thu Nov 14 15:15:09 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.9 >engine-health=vm-up good-health-status >gateway=True >mem-free=5197 >mem-load=0.00431910569106 > >MainThread::INFO::2013-11-14 15:15:09,593::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:15:09,602::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:15:09,603::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:15:09,603::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434909, 'last-update-local-ts': 1384434909.6027479, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:09,603::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:15:09,603::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434908, 'last-update-local-ts': 1384434909.6027479, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:09,604::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:15:19,672::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:15:19,926::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:15:20,020::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:15:20,080::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:15:20,081::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:15:20,081::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00444613821138 >MainThread::INFO::2013-11-14 15:15:20,081::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:20,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 5133 >MainThread::INFO::2013-11-14 15:15:20,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:15:20,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.76 >MainThread::INFO::2013-11-14 15:15:20,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:20,083::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434920|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434920 (Thu Nov 14 15:15:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.76 >engine-health=vm-up good-health-status >gateway=True >mem-free=5133 >mem-load=0.00444613821138 > >MainThread::INFO::2013-11-14 15:15:20,083::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:15:20,091::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:15:20,092::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:15:20,092::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434920, 'last-update-local-ts': 1384434920.091794, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:20,092::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:15:20,092::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434919, 'last-update-local-ts': 1384434920.091794, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:20,093::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:15:30,198::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:15:30,470::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:15:30,580::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:15:30,642::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:15:30,642::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:15:30,643::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00457317073171 >MainThread::INFO::2013-11-14 15:15:30,643::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:30,643::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 5061 >MainThread::INFO::2013-11-14 15:15:30,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:15:30,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.755 >MainThread::INFO::2013-11-14 15:15:30,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:30,645::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434930|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434930 (Thu Nov 14 15:15:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.755 >engine-health=vm-up good-health-status >gateway=True >mem-free=5061 >mem-load=0.00457317073171 > >MainThread::INFO::2013-11-14 15:15:30,646::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:15:30,652::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:15:30,653::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:15:30,653::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434930, 'last-update-local-ts': 1384434930.652853, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:30,653::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:15:30,653::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434929, 'last-update-local-ts': 1384434930.652853, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:30,654::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:15:40,828::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:15:41,102::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:15:41,202::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:15:41,308::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:15:41,317::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:15:41,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00470020325203 >MainThread::INFO::2013-11-14 15:15:41,342::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:41,342::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 5023 >MainThread::INFO::2013-11-14 15:15:41,343::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:15:41,343::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.635 >MainThread::INFO::2013-11-14 15:15:41,343::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:41,345::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434941|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434941 (Thu Nov 14 15:15:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.635 >engine-health=vm-up good-health-status >gateway=True >mem-free=5023 >mem-load=0.00470020325203 > >MainThread::INFO::2013-11-14 15:15:41,345::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:15:41,355::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:15:41,356::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:15:41,356::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434941, 'last-update-local-ts': 1384434941.3555419, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:41,356::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:15:41,356::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434940, 'last-update-local-ts': 1384434941.3555419, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:41,356::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:15:51,470::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:15:51,722::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:15:51,782::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:15:51,842::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:15:51,842::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:15:51,843::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00470020325203 >MainThread::INFO::2013-11-14 15:15:51,843::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:51,843::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4988 >MainThread::INFO::2013-11-14 15:15:51,844::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:15:51,844::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.575 >MainThread::INFO::2013-11-14 15:15:51,844::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:15:51,845::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434951|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434951 (Thu Nov 14 15:15:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.575 >engine-health=vm-up good-health-status >gateway=True >mem-free=4988 >mem-load=0.00470020325203 > >MainThread::INFO::2013-11-14 15:15:51,845::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:15:51,853::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:15:51,854::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:15:51,854::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434951, 'last-update-local-ts': 1384434951.85358, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:51,854::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:15:51,854::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434950, 'last-update-local-ts': 1384434951.85358, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:15:51,854::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:16:01,941::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:16:02,190::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:16:02,339::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:16:02,401::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:16:02,402::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:16:02,402::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00470020325203 >MainThread::INFO::2013-11-14 15:16:02,402::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:02,403::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4974 >MainThread::INFO::2013-11-14 15:16:02,403::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:16:02,403::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.635 >MainThread::INFO::2013-11-14 15:16:02,403::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:02,404::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434962|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434962 (Thu Nov 14 15:16:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.635 >engine-health=vm-up good-health-status >gateway=True >mem-free=4974 >mem-load=0.00470020325203 > >MainThread::INFO::2013-11-14 15:16:02,404::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:16:02,411::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:16:02,411::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:16:02,411::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434962, 'last-update-local-ts': 1384434962.411037, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:02,411::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:16:02,412::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434961, 'last-update-local-ts': 1384434962.411037, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:02,412::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:16:12,493::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:16:12,750::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:16:12,856::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:16:12,916::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:16:12,916::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:16:12,917::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00482723577236 >MainThread::INFO::2013-11-14 15:16:12,917::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:12,918::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4942 >MainThread::INFO::2013-11-14 15:16:12,918::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:16:12,918::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.535 >MainThread::INFO::2013-11-14 15:16:12,918::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:12,920::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434972|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434972 (Thu Nov 14 15:16:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.535 >engine-health=vm-up good-health-status >gateway=True >mem-free=4942 >mem-load=0.00482723577236 > >MainThread::INFO::2013-11-14 15:16:12,920::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:16:12,927::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:16:12,927::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:16:12,928::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434972, 'last-update-local-ts': 1384434972.927417, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:12,928::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:16:12,928::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434971, 'last-update-local-ts': 1384434972.927417, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:12,929::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:16:23,000::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:16:23,254::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:16:23,346::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:16:23,418::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:16:23,418::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:16:23,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00482723577236 >MainThread::INFO::2013-11-14 15:16:23,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:23,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4926 >MainThread::INFO::2013-11-14 15:16:23,420::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:16:23,420::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.455 >MainThread::INFO::2013-11-14 15:16:23,420::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:23,421::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434983|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434983 (Thu Nov 14 15:16:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.455 >engine-health=vm-up good-health-status >gateway=True >mem-free=4926 >mem-load=0.00482723577236 > >MainThread::INFO::2013-11-14 15:16:23,421::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:16:23,429::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:16:23,430::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:16:23,430::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434983, 'last-update-local-ts': 1384434983.4296269, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:23,430::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:16:23,430::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434981, 'last-update-local-ts': 1384434983.4296269, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:23,430::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:16:33,498::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:16:33,766::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:16:33,890::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:16:33,956::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:16:33,956::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:16:33,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00482723577236 >MainThread::INFO::2013-11-14 15:16:33,973::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:33,982::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4897 >MainThread::INFO::2013-11-14 15:16:33,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:16:33,995::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.385 >MainThread::INFO::2013-11-14 15:16:33,996::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:33,997::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384434993|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384434993 (Thu Nov 14 15:16:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.385 >engine-health=vm-up good-health-status >gateway=True >mem-free=4897 >mem-load=0.00482723577236 > >MainThread::INFO::2013-11-14 15:16:33,997::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:16:34,013::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:16:34,013::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:16:34,013::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384434993, 'last-update-local-ts': 1384434994.0130429, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:34,014::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:16:34,014::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384434992, 'last-update-local-ts': 1384434994.0130429, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:34,014::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:16:44,119::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:16:44,368::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:16:44,446::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:16:44,512::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:16:44,513::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:16:44,513::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00495426829268 >MainThread::INFO::2013-11-14 15:16:44,513::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:44,514::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4869 >MainThread::INFO::2013-11-14 15:16:44,514::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:16:44,514::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.325 >MainThread::INFO::2013-11-14 15:16:44,515::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:44,515::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435004|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435004 (Thu Nov 14 15:16:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.325 >engine-health=vm-up good-health-status >gateway=True >mem-free=4869 >mem-load=0.00495426829268 > >MainThread::INFO::2013-11-14 15:16:44,515::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:16:44,524::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:16:44,524::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:16:44,524::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435004, 'last-update-local-ts': 1384435004.5239971, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:44,524::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:16:44,525::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435002, 'last-update-local-ts': 1384435004.5239971, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:44,525::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:16:54,601::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:16:54,853::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:16:54,985::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:16:55,063::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:16:55,063::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:16:55,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00495426829268 >MainThread::INFO::2013-11-14 15:16:55,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:55,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4849 >MainThread::INFO::2013-11-14 15:16:55,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:16:55,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.315 >MainThread::INFO::2013-11-14 15:16:55,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:16:55,066::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435015|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435015 (Thu Nov 14 15:16:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.315 >engine-health=vm-up good-health-status >gateway=True >mem-free=4849 >mem-load=0.00495426829268 > >MainThread::INFO::2013-11-14 15:16:55,066::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:16:55,076::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:16:55,077::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:16:55,077::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435015, 'last-update-local-ts': 1384435015.0766151, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:55,077::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:16:55,077::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435013, 'last-update-local-ts': 1384435015.0766151, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:16:55,077::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:17:05,243::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:17:05,541::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:17:05,604::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:17:05,664::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:17:05,665::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:17:05,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00495426829268 >MainThread::INFO::2013-11-14 15:17:05,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:05,666::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4847 >MainThread::INFO::2013-11-14 15:17:05,666::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:17:05,666::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 15:17:05,666::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:05,667::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435025|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435025 (Thu Nov 14 15:17:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.265 >engine-health=vm-up good-health-status >gateway=True >mem-free=4847 >mem-load=0.00495426829268 > >MainThread::INFO::2013-11-14 15:17:05,667::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:17:05,675::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:17:05,676::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:17:05,676::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435025, 'last-update-local-ts': 1384435025.6758151, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:05,676::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:17:05,676::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435023, 'last-update-local-ts': 1384435025.6758151, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:05,677::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:17:15,751::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:17:16,007::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:17:16,188::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:17:16,251::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:17:16,252::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:17:16,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00495426829268 >MainThread::INFO::2013-11-14 15:17:16,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:16,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4821 >MainThread::INFO::2013-11-14 15:17:16,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:17:16,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 15:17:16,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:16,254::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435036|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435036 (Thu Nov 14 15:17:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-up good-health-status >gateway=True >mem-free=4821 >mem-load=0.00495426829268 > >MainThread::INFO::2013-11-14 15:17:16,254::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:17:16,266::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:17:16,266::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:17:16,267::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435036, 'last-update-local-ts': 1384435036.2664421, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:16,267::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:17:16,267::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435034, 'last-update-local-ts': 1384435036.2664421, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:16,267::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:17:26,372::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:17:26,626::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:17:26,726::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:17:26,791::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:17:26,791::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:17:26,792::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00495426829268 >MainThread::INFO::2013-11-14 15:17:26,792::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:26,793::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4817 >MainThread::INFO::2013-11-14 15:17:26,793::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:17:26,793::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.87 >MainThread::INFO::2013-11-14 15:17:26,794::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:26,794::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435046|1|2300|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435046 (Thu Nov 14 15:17:26 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.87 >engine-health=vm-up good-health-status >gateway=True >mem-free=4817 >mem-load=0.00495426829268 > >MainThread::INFO::2013-11-14 15:17:26,795::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:17:26,811::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:17:26,812::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:17:26,812::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435046, 'last-update-local-ts': 1384435046.811568, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:26,812::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:17:26,812::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435044, 'last-update-local-ts': 1384435046.811568, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:26,813::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:17:36,957::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:17:37,222::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:17:37,376::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:17:37,452::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:17:37,453::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:17:37,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00508130081301 >MainThread::INFO::2013-11-14 15:17:37,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:37,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4788 >MainThread::INFO::2013-11-14 15:17:37,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:17:37,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.78 >MainThread::INFO::2013-11-14 15:17:37,463::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:37,464::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435057|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435057 (Thu Nov 14 15:17:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.78 >engine-health=vm-up good-health-status >gateway=True >mem-free=4788 >mem-load=0.00508130081301 > >MainThread::INFO::2013-11-14 15:17:37,464::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:17:37,473::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:17:37,474::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:17:37,474::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435057, 'last-update-local-ts': 1384435057.4736869, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:37,474::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:17:37,474::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435055, 'last-update-local-ts': 1384435057.4736869, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:37,475::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:17:47,556::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:17:47,816::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:17:47,903::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:17:47,963::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:17:47,963::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:17:47,964::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00508130081301 >MainThread::INFO::2013-11-14 15:17:47,964::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:47,964::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4770 >MainThread::INFO::2013-11-14 15:17:47,965::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:17:47,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.14 >MainThread::INFO::2013-11-14 15:17:47,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:47,966::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435067|1|2300|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435067 (Thu Nov 14 15:17:47 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=1.14 >engine-health=vm-up good-health-status >gateway=True >mem-free=4770 >mem-load=0.00508130081301 > >MainThread::INFO::2013-11-14 15:17:47,967::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:17:47,977::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:17:47,977::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:17:47,978::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435067, 'last-update-local-ts': 1384435067.9773109, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:47,978::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:17:47,978::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435065, 'last-update-local-ts': 1384435067.9773109, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:47,978::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:17:58,083::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:17:58,330::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:17:58,431::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:17:58,514::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:17:58,514::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:17:58,514::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00508130081301 >MainThread::INFO::2013-11-14 15:17:58,515::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:58,515::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4769 >MainThread::INFO::2013-11-14 15:17:58,515::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:17:58,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.965 >MainThread::INFO::2013-11-14 15:17:58,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:17:58,517::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435078|1|2300|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435078 (Thu Nov 14 15:17:58 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.965 >engine-health=vm-up good-health-status >gateway=True >mem-free=4769 >mem-load=0.00508130081301 > >MainThread::INFO::2013-11-14 15:17:58,517::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:17:58,544::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:17:58,544::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:17:58,545::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435078, 'last-update-local-ts': 1384435078.5443859, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:58,545::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:17:58,545::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435076, 'last-update-local-ts': 1384435078.5443859, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:17:58,545::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:18:08,625::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:18:08,878::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:18:08,999::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:18:09,061::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:18:09,062::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:18:09,062::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00508130081301 >MainThread::INFO::2013-11-14 15:18:09,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:09,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4756 >MainThread::INFO::2013-11-14 15:18:09,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:18:09,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.815 >MainThread::INFO::2013-11-14 15:18:09,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:09,065::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435089|1|2300|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435089 (Thu Nov 14 15:18:09 2013) >host-id=1 >score=2300 >bridge=True >cpu-load=0.815 >engine-health=vm-up good-health-status >gateway=True >mem-free=4756 >mem-load=0.00508130081301 > >MainThread::INFO::2013-11-14 15:18:09,065::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:18:09,074::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:18:09,074::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:18:09,074::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435089, 'last-update-local-ts': 1384435089.073998, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2300, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:09,074::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:18:09,075::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435086, 'last-update-local-ts': 1384435089.073998, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:09,075::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:18:19,192::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:18:19,462::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:18:19,548::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:18:19,616::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:18:19,617::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:18:19,617::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00508130081301 >MainThread::INFO::2013-11-14 15:18:19,617::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:19,618::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4736 >MainThread::INFO::2013-11-14 15:18:19,618::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:18:19,618::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.77 >MainThread::INFO::2013-11-14 15:18:19,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:19,619::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435099|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435099 (Thu Nov 14 15:18:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.77 >engine-health=vm-up good-health-status >gateway=True >mem-free=4736 >mem-load=0.00508130081301 > >MainThread::INFO::2013-11-14 15:18:19,619::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:18:19,632::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:18:19,633::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:18:19,633::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435099, 'last-update-local-ts': 1384435099.6326931, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:19,633::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:18:19,633::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435096, 'last-update-local-ts': 1384435099.6326931, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:19,634::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:18:29,702::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:18:29,958::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:18:30,043::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:18:30,109::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:18:30,110::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:18:30,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00508130081301 >MainThread::INFO::2013-11-14 15:18:30,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:30,111::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4734 >MainThread::INFO::2013-11-14 15:18:30,111::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:18:30,111::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.655 >MainThread::INFO::2013-11-14 15:18:30,112::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:30,112::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435110|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435110 (Thu Nov 14 15:18:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.655 >engine-health=vm-up good-health-status >gateway=True >mem-free=4734 >mem-load=0.00508130081301 > >MainThread::INFO::2013-11-14 15:18:30,112::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:18:30,121::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:18:30,121::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:18:30,121::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435110, 'last-update-local-ts': 1384435110.121109, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:30,121::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:18:30,122::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435107, 'last-update-local-ts': 1384435110.121109, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:30,122::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:18:40,203::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:18:40,548::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:18:40,623::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:18:40,684::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:18:40,685::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:18:40,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00520833333333 >MainThread::INFO::2013-11-14 15:18:40,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:40,686::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4713 >MainThread::INFO::2013-11-14 15:18:40,686::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:18:40,686::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.55 >MainThread::INFO::2013-11-14 15:18:40,686::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:40,687::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435120|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435120 (Thu Nov 14 15:18:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.55 >engine-health=vm-up good-health-status >gateway=True >mem-free=4713 >mem-load=0.00520833333333 > >MainThread::INFO::2013-11-14 15:18:40,688::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:18:40,694::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:18:40,695::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:18:40,695::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435120, 'last-update-local-ts': 1384435120.694571, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:40,695::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:18:40,695::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435117, 'last-update-local-ts': 1384435120.694571, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:40,695::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:18:50,788::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:18:51,038::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:18:51,130::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:18:51,192::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:18:51,192::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:18:51,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00520833333333 >MainThread::INFO::2013-11-14 15:18:51,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:51,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4716 >MainThread::INFO::2013-11-14 15:18:51,194::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:18:51,194::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.465 >MainThread::INFO::2013-11-14 15:18:51,194::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:18:51,195::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435131|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435131 (Thu Nov 14 15:18:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.465 >engine-health=vm-up good-health-status >gateway=True >mem-free=4716 >mem-load=0.00520833333333 > >MainThread::INFO::2013-11-14 15:18:51,195::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:18:51,206::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:18:51,206::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:18:51,206::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435131, 'last-update-local-ts': 1384435131.206197, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:51,207::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:18:51,207::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435128, 'last-update-local-ts': 1384435131.206197, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:18:51,207::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:19:01,292::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:19:01,546::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:19:01,654::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:19:01,715::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:19:01,716::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:19:01,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00520833333333 >MainThread::INFO::2013-11-14 15:19:01,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:01,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4694 >MainThread::INFO::2013-11-14 15:19:01,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:19:01,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.395 >MainThread::INFO::2013-11-14 15:19:01,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:01,719::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435141|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435141 (Thu Nov 14 15:19:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.395 >engine-health=vm-up good-health-status >gateway=True >mem-free=4694 >mem-load=0.00520833333333 > >MainThread::INFO::2013-11-14 15:19:01,719::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:19:01,728::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:19:01,729::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:19:01,729::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435141, 'last-update-local-ts': 1384435141.72878, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:01,729::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:19:01,729::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435138, 'last-update-local-ts': 1384435141.72878, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:01,730::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:19:11,820::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:19:12,075::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:19:12,158::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:19:12,221::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:19:12,221::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:19:12,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00520833333333 >MainThread::INFO::2013-11-14 15:19:12,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:12,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4676 >MainThread::INFO::2013-11-14 15:19:12,223::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:19:12,223::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.495 >MainThread::INFO::2013-11-14 15:19:12,223::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:12,224::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435152|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435152 (Thu Nov 14 15:19:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.495 >engine-health=vm-up good-health-status >gateway=True >mem-free=4676 >mem-load=0.00520833333333 > >MainThread::INFO::2013-11-14 15:19:12,224::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:19:12,232::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:19:12,233::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:19:12,233::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435152, 'last-update-local-ts': 1384435152.232774, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:12,233::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:19:12,233::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435149, 'last-update-local-ts': 1384435152.232774, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:12,234::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:19:22,306::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:19:22,564::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:19:22,628::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:19:22,690::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:19:22,690::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:19:22,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00520833333333 >MainThread::INFO::2013-11-14 15:19:22,698::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:22,698::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4676 >MainThread::INFO::2013-11-14 15:19:22,698::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:19:22,699::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.42 >MainThread::INFO::2013-11-14 15:19:22,699::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:22,700::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435162|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435162 (Thu Nov 14 15:19:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.42 >engine-health=vm-up good-health-status >gateway=True >mem-free=4676 >mem-load=0.00520833333333 > >MainThread::INFO::2013-11-14 15:19:22,700::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:19:22,710::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:19:22,711::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:19:22,711::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435162, 'last-update-local-ts': 1384435162.7107129, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:22,711::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:19:22,711::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435159, 'last-update-local-ts': 1384435162.7107129, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:22,712::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:19:32,809::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:19:33,056::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:19:33,165::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:19:33,230::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:19:33,230::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:19:33,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00520833333333 >MainThread::INFO::2013-11-14 15:19:33,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:33,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4660 >MainThread::INFO::2013-11-14 15:19:33,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:19:33,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.355 >MainThread::INFO::2013-11-14 15:19:33,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:33,233::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435173|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435173 (Thu Nov 14 15:19:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.355 >engine-health=vm-up good-health-status >gateway=True >mem-free=4660 >mem-load=0.00520833333333 > >MainThread::INFO::2013-11-14 15:19:33,233::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:19:33,242::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:19:33,242::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:19:33,242::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435173, 'last-update-local-ts': 1384435173.2419319, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:33,242::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:19:33,242::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435170, 'last-update-local-ts': 1384435173.2419319, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:33,243::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:19:43,332::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:19:43,581::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:19:43,681::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:19:43,744::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:19:43,744::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:19:43,750::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00520833333333 >MainThread::INFO::2013-11-14 15:19:43,751::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:43,751::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4646 >MainThread::INFO::2013-11-14 15:19:43,751::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:19:43,752::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.42 >MainThread::INFO::2013-11-14 15:19:43,752::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:43,752::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435183|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435183 (Thu Nov 14 15:19:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.42 >engine-health=vm-up good-health-status >gateway=True >mem-free=4646 >mem-load=0.00520833333333 > >MainThread::INFO::2013-11-14 15:19:43,753::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:19:43,761::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:19:43,761::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:19:43,762::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435183, 'last-update-local-ts': 1384435183.761411, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:43,762::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:19:43,762::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435180, 'last-update-local-ts': 1384435183.761411, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:43,762::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:19:53,905::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:19:54,169::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:19:54,239::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:19:54,301::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:19:54,301::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:19:54,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00533536585366 >MainThread::INFO::2013-11-14 15:19:54,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:54,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4625 >MainThread::INFO::2013-11-14 15:19:54,303::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:19:54,303::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.39 >MainThread::INFO::2013-11-14 15:19:54,303::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:19:54,304::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435194|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435194 (Thu Nov 14 15:19:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.39 >engine-health=vm-up good-health-status >gateway=True >mem-free=4625 >mem-load=0.00533536585366 > >MainThread::INFO::2013-11-14 15:19:54,305::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:19:54,313::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:19:54,314::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:19:54,314::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435194, 'last-update-local-ts': 1384435194.313719, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:54,314::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:19:54,314::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435191, 'last-update-local-ts': 1384435194.313719, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:19:54,315::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:20:04,387::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:20:04,657::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:20:04,718::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:20:04,779::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:20:04,780::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:20:04,786::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00533536585366 >MainThread::INFO::2013-11-14 15:20:04,786::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:04,787::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4608 >MainThread::INFO::2013-11-14 15:20:04,787::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:20:04,787::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.33 >MainThread::INFO::2013-11-14 15:20:04,788::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:04,788::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435204|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435204 (Thu Nov 14 15:20:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.33 >engine-health=vm-up good-health-status >gateway=True >mem-free=4608 >mem-load=0.00533536585366 > >MainThread::INFO::2013-11-14 15:20:04,788::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:20:04,796::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:20:04,796::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:20:04,797::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435204, 'last-update-local-ts': 1384435204.7965159, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:04,797::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:20:04,797::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435201, 'last-update-local-ts': 1384435204.7965159, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:04,797::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:20:14,900::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:20:15,236::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:20:15,336::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:20:15,404::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:20:15,404::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:20:15,405::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00533536585366 >MainThread::INFO::2013-11-14 15:20:15,405::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:15,405::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4593 >MainThread::INFO::2013-11-14 15:20:15,406::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:20:15,406::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.6 >MainThread::INFO::2013-11-14 15:20:15,406::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:15,407::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435215|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435215 (Thu Nov 14 15:20:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.6 >engine-health=vm-up good-health-status >gateway=True >mem-free=4593 >mem-load=0.00533536585366 > >MainThread::INFO::2013-11-14 15:20:15,407::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:20:15,413::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:20:15,414::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:20:15,414::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435215, 'last-update-local-ts': 1384435215.413873, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:15,414::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:20:15,414::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435212, 'last-update-local-ts': 1384435215.413873, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:15,415::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:20:25,519::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:20:25,842::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:20:25,928::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:20:25,990::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:20:25,990::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:20:25,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00533536585366 >MainThread::INFO::2013-11-14 15:20:25,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:25,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4591 >MainThread::INFO::2013-11-14 15:20:25,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:20:25,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.51 >MainThread::INFO::2013-11-14 15:20:25,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:25,993::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435225|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435225 (Thu Nov 14 15:20:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.51 >engine-health=vm-up good-health-status >gateway=True >mem-free=4591 >mem-load=0.00533536585366 > >MainThread::INFO::2013-11-14 15:20:25,993::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:20:26,001::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:20:26,001::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:20:26,001::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435225, 'last-update-local-ts': 1384435226.001199, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:26,002::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:20:26,002::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435222, 'last-update-local-ts': 1384435226.001199, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:26,002::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:20:36,100::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:20:36,359::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:20:36,438::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:20:36,528::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:20:36,528::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:20:36,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00533536585366 >MainThread::INFO::2013-11-14 15:20:36,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:36,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4577 >MainThread::INFO::2013-11-14 15:20:36,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:20:36,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.47 >MainThread::INFO::2013-11-14 15:20:36,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:36,530::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435236|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435236 (Thu Nov 14 15:20:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.47 >engine-health=vm-up good-health-status >gateway=True >mem-free=4577 >mem-load=0.00533536585366 > >MainThread::INFO::2013-11-14 15:20:36,531::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:20:36,537::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:20:36,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:20:36,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435236, 'last-update-local-ts': 1384435236.5378799, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:36,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:20:36,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435232, 'last-update-local-ts': 1384435236.5378799, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:36,539::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:20:46,623::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:20:46,899::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:20:46,985::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:20:47,048::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:20:47,048::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:20:47,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00533536585366 >MainThread::INFO::2013-11-14 15:20:47,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:47,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4573 >MainThread::INFO::2013-11-14 15:20:47,056::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:20:47,056::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.395 >MainThread::INFO::2013-11-14 15:20:47,056::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:47,057::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435247|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435247 (Thu Nov 14 15:20:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.395 >engine-health=vm-up good-health-status >gateway=True >mem-free=4573 >mem-load=0.00533536585366 > >MainThread::INFO::2013-11-14 15:20:47,057::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:20:47,068::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:20:47,068::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:20:47,069::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435247, 'last-update-local-ts': 1384435247.068223, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:47,069::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:20:47,069::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435243, 'last-update-local-ts': 1384435247.068223, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:47,069::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:20:57,152::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:20:57,409::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:20:57,466::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:20:57,526::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:20:57,526::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:20:57,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:20:57,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:57,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4555 >MainThread::INFO::2013-11-14 15:20:57,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:20:57,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 15:20:57,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:20:57,528::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435257|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435257 (Thu Nov 14 15:20:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-up good-health-status >gateway=True >mem-free=4555 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:20:57,529::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:20:57,537::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:20:57,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:20:57,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435257, 'last-update-local-ts': 1384435257.5375879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:57,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:20:57,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435253, 'last-update-local-ts': 1384435257.5375879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:20:57,538::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:21:07,625::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:21:07,888::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:21:08,005::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:21:08,066::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:21:08,066::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:21:08,067::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:21:08,067::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:08,067::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4554 >MainThread::INFO::2013-11-14 15:21:08,068::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:21:08,068::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 15:21:08,068::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:08,069::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435268|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435268 (Thu Nov 14 15:21:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-up good-health-status >gateway=True >mem-free=4554 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:21:08,069::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:21:08,078::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:21:08,078::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:21:08,078::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435268, 'last-update-local-ts': 1384435268.0779409, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:08,078::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:21:08,079::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435264, 'last-update-local-ts': 1384435268.0779409, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:08,079::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:21:18,159::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:21:18,409::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:21:18,522::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:21:18,584::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:21:18,584::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:21:18,585::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:21:18,585::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:18,585::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4555 >MainThread::INFO::2013-11-14 15:21:18,585::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:21:18,586::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.275 >MainThread::INFO::2013-11-14 15:21:18,586::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:18,587::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435278|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435278 (Thu Nov 14 15:21:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.275 >engine-health=vm-up good-health-status >gateway=True >mem-free=4555 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:21:18,587::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:21:18,597::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:21:18,598::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:21:18,598::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435278, 'last-update-local-ts': 1384435278.5976069, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:18,598::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:21:18,598::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435274, 'last-update-local-ts': 1384435278.5976069, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:18,598::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:21:28,674::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:21:28,922::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:21:28,992::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:21:29,053::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:21:29,053::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:21:29,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:21:29,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:29,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4552 >MainThread::INFO::2013-11-14 15:21:29,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:21:29,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 15:21:29,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:29,056::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435289|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435289 (Thu Nov 14 15:21:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-up good-health-status >gateway=True >mem-free=4552 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:21:29,056::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:21:29,064::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:21:29,065::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:21:29,065::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435289, 'last-update-local-ts': 1384435289.064851, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:29,065::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:21:29,065::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435285, 'last-update-local-ts': 1384435289.064851, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:29,066::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:21:39,146::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:21:39,399::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:21:39,489::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:21:39,549::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:21:39,550::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:21:39,550::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:21:39,550::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:39,551::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4532 >MainThread::INFO::2013-11-14 15:21:39,551::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:21:39,551::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 15:21:39,552::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:39,552::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435299|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435299 (Thu Nov 14 15:21:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-up good-health-status >gateway=True >mem-free=4532 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:21:39,552::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:21:39,562::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:21:39,562::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:21:39,563::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435299, 'last-update-local-ts': 1384435299.562407, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:39,563::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:21:39,563::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435295, 'last-update-local-ts': 1384435299.562407, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:39,563::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:21:49,659::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:21:49,910::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:21:49,993::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:21:50,061::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:21:50,061::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:21:50,062::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:21:50,062::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:50,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4533 >MainThread::INFO::2013-11-14 15:21:50,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:21:50,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 15:21:50,063::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:21:50,065::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435310|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435310 (Thu Nov 14 15:21:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-up good-health-status >gateway=True >mem-free=4533 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:21:50,065::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:21:50,078::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:21:50,078::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:21:50,078::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435310, 'last-update-local-ts': 1384435310.0780351, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:50,078::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:21:50,079::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435306, 'last-update-local-ts': 1384435310.0780351, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:21:50,079::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:22:00,163::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:22:00,422::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:22:00,503::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:22:00,582::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:22:00,582::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:22:00,584::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:22:00,585::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:00,585::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4513 >MainThread::INFO::2013-11-14 15:22:00,586::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:22:00,589::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.28 >MainThread::INFO::2013-11-14 15:22:00,589::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:00,590::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435320|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435320 (Thu Nov 14 15:22:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.28 >engine-health=vm-up good-health-status >gateway=True >mem-free=4513 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:22:00,590::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:22:00,605::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:22:00,605::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:22:00,606::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435320, 'last-update-local-ts': 1384435320.6049809, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:00,606::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:22:00,606::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435316, 'last-update-local-ts': 1384435320.6049809, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:00,606::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:22:10,699::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:22:10,952::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:22:11,076::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:22:11,138::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:22:11,138::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:22:11,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:22:11,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:11,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4520 >MainThread::INFO::2013-11-14 15:22:11,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:22:11,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 15:22:11,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:11,141::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435331|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435331 (Thu Nov 14 15:22:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-up good-health-status >gateway=True >mem-free=4520 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:22:11,142::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:22:11,151::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:22:11,152::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:22:11,152::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435331, 'last-update-local-ts': 1384435331.151489, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:11,152::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:22:11,152::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435327, 'last-update-local-ts': 1384435331.151489, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:11,152::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:22:21,236::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:22:21,485::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:22:21,597::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:22:21,658::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:22:21,658::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:22:21,664::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:22:21,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:21,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4508 >MainThread::INFO::2013-11-14 15:22:21,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:22:21,666::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 15:22:21,666::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:21,666::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435341|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435341 (Thu Nov 14 15:22:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-up good-health-status >gateway=True >mem-free=4508 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:22:21,667::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:22:21,677::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:22:21,677::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:22:21,678::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435341, 'last-update-local-ts': 1384435341.6773679, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:21,678::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:22:21,678::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435337, 'last-update-local-ts': 1384435341.6773679, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:21,678::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:22:31,778::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:22:32,034::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:22:32,130::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:22:32,191::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:22:32,192::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:22:32,192::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:22:32,192::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:32,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4504 >MainThread::INFO::2013-11-14 15:22:32,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:22:32,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.355 >MainThread::INFO::2013-11-14 15:22:32,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:32,195::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435352|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435352 (Thu Nov 14 15:22:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.355 >engine-health=vm-up good-health-status >gateway=True >mem-free=4504 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:22:32,195::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:22:32,203::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:22:32,204::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:22:32,204::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435352, 'last-update-local-ts': 1384435352.2038269, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:32,204::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:22:32,204::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435347, 'last-update-local-ts': 1384435352.2038269, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:32,205::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:22:42,279::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:22:42,524::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:22:42,603::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:22:42,664::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:22:42,664::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:22:42,670::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:22:42,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:42,671::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4504 >MainThread::INFO::2013-11-14 15:22:42,672::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:22:42,672::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 15:22:42,672::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:42,673::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435362|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435362 (Thu Nov 14 15:22:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-up good-health-status >gateway=True >mem-free=4504 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:22:42,673::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:22:42,682::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:22:42,682::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:22:42,683::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435362, 'last-update-local-ts': 1384435362.6824551, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:42,683::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:22:42,683::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435358, 'last-update-local-ts': 1384435362.6824551, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:42,683::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:22:52,765::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:22:53,021::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:22:53,116::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:22:53,177::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:22:53,177::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:22:53,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:22:53,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:53,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4498 >MainThread::INFO::2013-11-14 15:22:53,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:22:53,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.62 >MainThread::INFO::2013-11-14 15:22:53,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:22:53,180::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435373|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435373 (Thu Nov 14 15:22:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.62 >engine-health=vm-up good-health-status >gateway=True >mem-free=4498 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:22:53,180::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:22:53,189::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:22:53,189::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:22:53,189::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435373, 'last-update-local-ts': 1384435373.1890831, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:53,189::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:22:53,190::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435368, 'last-update-local-ts': 1384435373.1890831, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:22:53,190::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:23:03,272::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:23:03,521::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:23:03,618::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:23:03,679::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:23:03,680::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:23:03,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00546239837398 >MainThread::INFO::2013-11-14 15:23:03,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:03,681::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4488 >MainThread::INFO::2013-11-14 15:23:03,681::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:23:03,681::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.525 >MainThread::INFO::2013-11-14 15:23:03,681::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:03,682::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435383|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435383 (Thu Nov 14 15:23:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.525 >engine-health=vm-up good-health-status >gateway=True >mem-free=4488 >mem-load=0.00546239837398 > >MainThread::INFO::2013-11-14 15:23:03,682::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:23:03,691::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:23:03,691::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:23:03,691::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435383, 'last-update-local-ts': 1384435383.6909189, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:03,691::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:23:03,691::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435379, 'last-update-local-ts': 1384435383.6909189, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:03,692::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:23:13,760::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:23:14,111::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:23:14,386::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:23:14,453::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:23:14,453::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:23:14,454::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:23:14,454::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:14,454::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4486 >MainThread::INFO::2013-11-14 15:23:14,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:23:14,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.445 >MainThread::INFO::2013-11-14 15:23:14,455::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:14,457::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435394|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435394 (Thu Nov 14 15:23:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.445 >engine-health=vm-up good-health-status >gateway=True >mem-free=4486 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:23:14,457::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:23:14,465::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:23:14,465::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:23:14,466::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435394, 'last-update-local-ts': 1384435394.46527, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:14,466::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:23:14,466::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435389, 'last-update-local-ts': 1384435394.46527, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:14,466::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:23:24,548::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:23:24,826::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:23:24,880::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:23:24,941::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:23:24,941::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:23:24,942::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:23:24,942::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:24,942::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4476 >MainThread::INFO::2013-11-14 15:23:24,943::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:23:24,943::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.375 >MainThread::INFO::2013-11-14 15:23:24,943::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:24,944::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435404|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435404 (Thu Nov 14 15:23:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.375 >engine-health=vm-up good-health-status >gateway=True >mem-free=4476 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:23:24,944::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:23:24,952::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:23:24,953::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:23:24,953::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435404, 'last-update-local-ts': 1384435404.9526751, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:24,953::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:23:24,953::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435400, 'last-update-local-ts': 1384435404.9526751, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:24,954::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:23:35,040::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:23:35,350::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:23:35,478::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:23:35,539::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:23:35,539::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:23:35,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:23:35,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:35,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4466 >MainThread::INFO::2013-11-14 15:23:35,547::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:23:35,547::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 15:23:35,547::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:35,548::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435415|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435415 (Thu Nov 14 15:23:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.32 >engine-health=vm-up good-health-status >gateway=True >mem-free=4466 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:23:35,548::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:23:35,557::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:23:35,557::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:23:35,558::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435415, 'last-update-local-ts': 1384435415.5573871, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:35,558::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:23:35,558::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435410, 'last-update-local-ts': 1384435415.5573871, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:35,558::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:23:45,643::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:23:46,047::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:23:46,110::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:23:46,170::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:23:46,171::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:23:46,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:23:46,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:46,172::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4466 >MainThread::INFO::2013-11-14 15:23:46,172::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:23:46,172::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.27 >MainThread::INFO::2013-11-14 15:23:46,173::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:46,173::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435426|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435426 (Thu Nov 14 15:23:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.27 >engine-health=vm-up good-health-status >gateway=True >mem-free=4466 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:23:46,173::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:23:46,182::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:23:46,182::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:23:46,182::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435426, 'last-update-local-ts': 1384435426.182225, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:46,183::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:23:46,183::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435420, 'last-update-local-ts': 1384435426.182225, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:46,183::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:23:56,261::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:23:56,517::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:23:56,602::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:23:56,664::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:23:56,664::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:23:56,664::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:23:56,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:56,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4454 >MainThread::INFO::2013-11-14 15:23:56,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:23:56,666::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 15:23:56,666::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:23:56,666::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435436|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435436 (Thu Nov 14 15:23:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-up good-health-status >gateway=True >mem-free=4454 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:23:56,667::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:23:56,675::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:23:56,675::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:23:56,676::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435436, 'last-update-local-ts': 1384435436.6754489, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:56,676::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:23:56,676::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435431, 'last-update-local-ts': 1384435436.6754489, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:23:56,676::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:24:06,763::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:24:07,017::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:24:07,131::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:24:07,193::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:24:07,193::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:24:07,193::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:24:07,194::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:07,194::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4452 >MainThread::INFO::2013-11-14 15:24:07,194::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:24:07,195::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 15:24:07,195::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:07,196::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435447|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435447 (Thu Nov 14 15:24:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-up good-health-status >gateway=True >mem-free=4452 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:24:07,196::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:24:07,205::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:24:07,205::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:24:07,206::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435447, 'last-update-local-ts': 1384435447.2054651, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:07,206::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:24:07,206::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435441, 'last-update-local-ts': 1384435447.2054651, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:07,206::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:24:17,284::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:24:17,532::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:24:17,627::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:24:17,688::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:24:17,689::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:24:17,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:24:17,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:17,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4450 >MainThread::INFO::2013-11-14 15:24:17,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:24:17,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 15:24:17,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:17,691::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435457|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435457 (Thu Nov 14 15:24:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.265 >engine-health=vm-up good-health-status >gateway=True >mem-free=4450 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:24:17,691::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:24:17,701::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:24:17,701::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:24:17,701::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435457, 'last-update-local-ts': 1384435457.7011089, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:17,701::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:24:17,702::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435452, 'last-update-local-ts': 1384435457.7011089, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:17,702::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:24:27,777::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:24:28,050::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:24:28,142::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:24:28,204::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:24:28,204::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:24:28,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:24:28,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:28,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4449 >MainThread::INFO::2013-11-14 15:24:28,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:24:28,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 15:24:28,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:28,207::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435468|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435468 (Thu Nov 14 15:24:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-up good-health-status >gateway=True >mem-free=4449 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:24:28,208::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:24:28,216::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:24:28,217::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:24:28,217::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435468, 'last-update-local-ts': 1384435468.2168491, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:28,217::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:24:28,217::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435462, 'last-update-local-ts': 1384435468.2168491, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:28,218::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:24:38,361::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:24:38,626::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:24:38,706::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:24:38,766::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:24:38,767::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:24:38,767::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:24:38,767::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:38,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4447 >MainThread::INFO::2013-11-14 15:24:38,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:24:38,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 15:24:38,769::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:38,769::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435478|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435478 (Thu Nov 14 15:24:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-up good-health-status >gateway=True >mem-free=4447 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:24:38,769::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:24:38,786::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:24:38,787::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:24:38,787::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435478, 'last-update-local-ts': 1384435478.7867539, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:38,787::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:24:38,787::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435473, 'last-update-local-ts': 1384435478.7867539, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:38,788::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:24:48,875::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:24:49,140::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:24:49,243::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:24:49,304::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:24:49,304::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:24:49,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:24:49,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:49,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4443 >MainThread::INFO::2013-11-14 15:24:49,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:24:49,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 15:24:49,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:49,307::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435489|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435489 (Thu Nov 14 15:24:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-up good-health-status >gateway=True >mem-free=4443 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:24:49,308::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:24:49,316::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:24:49,317::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:24:49,317::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435489, 'last-update-local-ts': 1384435489.3165901, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:49,317::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:24:49,317::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435483, 'last-update-local-ts': 1384435489.3165901, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:49,317::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:24:59,416::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:24:59,672::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:24:59,753::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:24:59,814::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:24:59,815::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:24:59,815::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:24:59,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:59,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4435 >MainThread::INFO::2013-11-14 15:24:59,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:24:59,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 15:24:59,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:24:59,817::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435499|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435499 (Thu Nov 14 15:24:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-up good-health-status >gateway=True >mem-free=4435 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:24:59,817::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:24:59,825::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:24:59,826::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:24:59,826::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435499, 'last-update-local-ts': 1384435499.8258879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:59,826::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:24:59,826::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435494, 'last-update-local-ts': 1384435499.8258879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:24:59,827::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:25:09,911::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:25:10,166::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:25:10,224::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:25:10,284::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:25:10,285::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:25:10,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:25:10,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:10,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4435 >MainThread::INFO::2013-11-14 15:25:10,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:25:10,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:25:10,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:10,288::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435510|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435510 (Thu Nov 14 15:25:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-up good-health-status >gateway=True >mem-free=4435 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:25:10,288::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:25:10,302::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:25:10,302::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:25:10,303::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435510, 'last-update-local-ts': 1384435510.3023641, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:10,303::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:25:10,303::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435504, 'last-update-local-ts': 1384435510.3023641, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:10,303::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:25:20,400::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:25:20,681::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:25:20,802::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:25:20,863::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:25:20,863::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:25:20,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:25:20,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:20,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4418 >MainThread::INFO::2013-11-14 15:25:20,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:25:20,865::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 15:25:20,865::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:20,865::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435520|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435520 (Thu Nov 14 15:25:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-up good-health-status >gateway=True >mem-free=4418 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:25:20,866::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:25:20,872::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:25:20,873::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:25:20,873::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435520, 'last-update-local-ts': 1384435520.872551, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:20,873::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:25:20,873::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435515, 'last-update-local-ts': 1384435520.872551, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:20,873::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:25:30,961::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:25:31,216::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:25:31,329::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:25:31,391::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:25:31,391::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:25:31,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:25:31,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:31,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4428 >MainThread::INFO::2013-11-14 15:25:31,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:25:31,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:25:31,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:31,394::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435531|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435531 (Thu Nov 14 15:25:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-up good-health-status >gateway=True >mem-free=4428 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:25:31,394::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:25:31,401::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:25:31,401::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:25:31,402::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435531, 'last-update-local-ts': 1384435531.4013591, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:31,402::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:25:31,402::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435525, 'last-update-local-ts': 1384435531.4013591, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:31,402::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:25:41,648::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:25:42,141::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:25:42,392::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:25:42,487::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:25:42,487::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:25:42,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00558943089431 >MainThread::INFO::2013-11-14 15:25:42,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:42,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 4399 >MainThread::INFO::2013-11-14 15:25:42,490::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up good-health-status >MainThread::INFO::2013-11-14 15:25:42,490::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 15:25:42,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:42,493::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435542|1|2400|vm-up good-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435542 (Thu Nov 14 15:25:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-up good-health-status >gateway=True >mem-free=4399 >mem-load=0.00558943089431 > >MainThread::INFO::2013-11-14 15:25:42,493::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:25:42,508::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:25:42,511::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:25:42,512::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435542, 'last-update-local-ts': 1384435542.5081029, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:42,512::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:25:42,512::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435535, 'last-update-local-ts': 1384435542.5081029, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:42,513::hosted_engine::1021::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-14 15:25:52,647::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:25:52,971::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:25:53,058::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:25:53,142::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:25:53,142::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:25:53,143::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00647865853659 >MainThread::INFO::2013-11-14 15:25:53,143::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:53,144::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 3742 >MainThread::INFO::2013-11-14 15:25:53,144::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:25:53,144::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.395 >MainThread::INFO::2013-11-14 15:25:53,145::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:25:53,145::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435553|1|2000|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435553 (Thu Nov 14 15:25:53 2013) >host-id=1 >score=2000 >bridge=True >cpu-load=0.395 >engine-health=vm-up bad-health-status >gateway=True >mem-free=3742 >mem-load=0.00647865853659 > >MainThread::INFO::2013-11-14 15:25:53,145::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:25:53,157::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:25:53,157::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:25:53,158::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435553, 'last-update-local-ts': 1384435553.15692, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2000, 'first-update': False} >MainThread::INFO::2013-11-14 15:25:53,158::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:25:53,158::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435546, 'last-update-local-ts': 1384435553.15692, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::ERROR::2013-11-14 15:25:53,158::hosted_engine::1016::HostedEngine::(_handle_on) Engine VM timed out with bad health status after 300 seconds, restarting >MainThread::INFO::2013-11-14 15:25:53,159::hosted_engine::1067::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-14 15:26:23,059::hosted_engine::1071::HostedEngine::(_stop_engine_vm) stdout: Failed to shutdown VM. Try Forced Shutdown. > >MainThread::INFO::2013-11-14 15:26:23,060::hosted_engine::1072::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-14 15:26:23,060::hosted_engine::1077::HostedEngine::(_stop_engine_vm) Failed to stop engine vm with /usr/sbin/hosted-engine --vm-shutdown: >MainThread::ERROR::2013-11-14 15:26:23,060::hosted_engine::1048::HostedEngine::(_handle_stop) Failed to stop engine VM: >MainThread::INFO::2013-11-14 15:26:33,228::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:26:33,585::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:26:33,695::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:26:33,764::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:26:33,764::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:26:33,765::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00825711382114 >MainThread::INFO::2013-11-14 15:26:33,766::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:26:33,766::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 2790 >MainThread::INFO::2013-11-14 15:26:33,767::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:26:33,767::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.885 >MainThread::INFO::2013-11-14 15:26:33,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:26:33,769::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435593|1|0|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435593 (Thu Nov 14 15:26:33 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.885 >engine-health=vm-up bad-health-status >gateway=True >mem-free=2790 >mem-load=0.00825711382114 > >MainThread::INFO::2013-11-14 15:26:33,770::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:26:33,796::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:26:33,796::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:26:33,796::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435593, 'last-update-local-ts': 1384435593.795857, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:26:33,797::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:26:33,797::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435588, 'last-update-local-ts': 1384435593.795857, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:26:33,797::hosted_engine::831::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::ERROR::2013-11-14 15:26:33,798::hosted_engine::998::HostedEngine::(_handle_on) Host master-vds10.qa.lab.tlv.redhat.com (id 2) score is significantly better than local score, migrating vm >MainThread::ERROR::2013-11-14 15:26:33,864::hosted_engine::1104::HostedEngine::(_handle_migrate) Failed to start migration >Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 1101, in _handle_migrate > dst=self._all_host_stats[best_host_id]['hostname'], > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/vds_client.py", line 85, in run_vds_client_cmd > response['status']['message']) >DetailedError: Error 4 from migrate: Virtual machine already exists >MainThread::ERROR::2013-11-14 15:26:33,875::hosted_engine::1147::HostedEngine::(_handle_migrate) Migration to host master-vds10.qa.lab.tlv.redhat.com (id 2) failed >MainThread::INFO::2013-11-14 15:26:33,876::hosted_engine::1067::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-14 15:27:04,088::hosted_engine::1071::HostedEngine::(_stop_engine_vm) stdout: Failed to shutdown VM. Try Forced Shutdown. > >MainThread::INFO::2013-11-14 15:27:04,090::hosted_engine::1072::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-14 15:27:04,090::hosted_engine::1077::HostedEngine::(_stop_engine_vm) Failed to stop engine vm with /usr/sbin/hosted-engine --vm-shutdown: >MainThread::ERROR::2013-11-14 15:27:04,091::hosted_engine::1048::HostedEngine::(_handle_stop) Failed to stop engine VM: >MainThread::INFO::2013-11-14 15:27:14,245::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:27:14,554::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:27:14,698::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:27:14,763::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:27:14,764::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:27:14,777::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00825711382114 >MainThread::INFO::2013-11-14 15:27:14,777::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:27:14,778::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 2788 >MainThread::INFO::2013-11-14 15:27:14,778::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:27:14,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.205 >MainThread::INFO::2013-11-14 15:27:14,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:27:14,779::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435634|1|0|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435634 (Thu Nov 14 15:27:14 2013) >host-id=1 >score=0 >bridge=True >cpu-load=1.205 >engine-health=vm-up bad-health-status >gateway=True >mem-free=2788 >mem-load=0.00825711382114 > >MainThread::INFO::2013-11-14 15:27:14,780::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:27:14,803::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:27:14,804::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:27:14,804::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435634, 'last-update-local-ts': 1384435634.803493, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:27:14,804::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:27:14,804::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435630, 'last-update-local-ts': 1384435634.803493, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:27:14,805::hosted_engine::831::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::ERROR::2013-11-14 15:27:14,805::hosted_engine::998::HostedEngine::(_handle_on) Host master-vds10.qa.lab.tlv.redhat.com (id 2) score is significantly better than local score, migrating vm >MainThread::ERROR::2013-11-14 15:27:14,879::hosted_engine::1104::HostedEngine::(_handle_migrate) Failed to start migration >Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 1101, in _handle_migrate > dst=self._all_host_stats[best_host_id]['hostname'], > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/vds_client.py", line 85, in run_vds_client_cmd > response['status']['message']) >DetailedError: Error 4 from migrate: Virtual machine already exists >MainThread::ERROR::2013-11-14 15:27:14,880::hosted_engine::1147::HostedEngine::(_handle_migrate) Migration to host master-vds10.qa.lab.tlv.redhat.com (id 2) failed >MainThread::INFO::2013-11-14 15:27:14,880::hosted_engine::1067::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-14 15:27:45,063::hosted_engine::1071::HostedEngine::(_stop_engine_vm) stdout: Failed to shutdown VM. Try Forced Shutdown. > >MainThread::INFO::2013-11-14 15:27:45,064::hosted_engine::1072::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-14 15:27:45,064::hosted_engine::1077::HostedEngine::(_stop_engine_vm) Failed to stop engine vm with /usr/sbin/hosted-engine --vm-shutdown: >MainThread::ERROR::2013-11-14 15:27:45,064::hosted_engine::1048::HostedEngine::(_handle_stop) Failed to stop engine VM: >MainThread::INFO::2013-11-14 15:27:55,185::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:27:55,642::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:27:55,774::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:27:55,849::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:27:55,850::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:27:55,866::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00825711382114 >MainThread::INFO::2013-11-14 15:27:55,866::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:27:55,867::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 2782 >MainThread::INFO::2013-11-14 15:27:55,867::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 15:27:55,867::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.44 >MainThread::INFO::2013-11-14 15:27:55,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:27:55,869::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435675|1|0|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435675 (Thu Nov 14 15:27:55 2013) >host-id=1 >score=0 >bridge=True >cpu-load=1.44 >engine-health=vm-up bad-health-status >gateway=True >mem-free=2782 >mem-load=0.00825711382114 > >MainThread::INFO::2013-11-14 15:27:55,869::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:27:55,882::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:27:55,882::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:27:55,883::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435675, 'last-update-local-ts': 1384435675.882051, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:27:55,883::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:27:55,883::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435671, 'last-update-local-ts': 1384435675.882051, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:27:55,884::hosted_engine::831::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::ERROR::2013-11-14 15:27:55,884::hosted_engine::998::HostedEngine::(_handle_on) Host master-vds10.qa.lab.tlv.redhat.com (id 2) score is significantly better than local score, migrating vm >MainThread::ERROR::2013-11-14 15:27:55,953::hosted_engine::1104::HostedEngine::(_handle_migrate) Failed to start migration >Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 1101, in _handle_migrate > dst=self._all_host_stats[best_host_id]['hostname'], > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/vds_client.py", line 85, in run_vds_client_cmd > response['status']['message']) >DetailedError: Error 4 from migrate: Virtual machine already exists >MainThread::ERROR::2013-11-14 15:27:55,954::hosted_engine::1147::HostedEngine::(_handle_migrate) Migration to host master-vds10.qa.lab.tlv.redhat.com (id 2) failed >MainThread::INFO::2013-11-14 15:27:55,954::hosted_engine::1067::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-14 15:28:13,897::hosted_engine::1071::HostedEngine::(_stop_engine_vm) stdout: Failed to shutdown VM. Try Forced Shutdown. > >MainThread::INFO::2013-11-14 15:28:13,898::hosted_engine::1072::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-14 15:28:13,898::hosted_engine::1077::HostedEngine::(_stop_engine_vm) Failed to stop engine vm with /usr/sbin/hosted-engine --vm-shutdown: >MainThread::ERROR::2013-11-14 15:28:13,899::hosted_engine::1048::HostedEngine::(_handle_stop) Failed to stop engine VM: >MainThread::INFO::2013-11-14 15:28:23,991::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:28:24,246::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:28:24,380::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:28:24,447::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:28:24,447::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:28:24,448::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:28:24,448::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:28:24,448::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:28:24,449::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:28:24,449::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.44 >MainThread::INFO::2013-11-14 15:28:24,449::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:28:24,450::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435704|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435704 (Thu Nov 14 15:28:24 2013) >host-id=1 >score=0 >bridge=True >cpu-load=1.44 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:28:24,451::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:28:24,460::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:28:24,460::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:28:24,460::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435704, 'last-update-local-ts': 1384435704.460247, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:28:24,461::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:28:24,461::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435703, 'last-update-local-ts': 1384435704.460247, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:28:24,461::hosted_engine::831::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::INFO::2013-11-14 15:28:24,461::hosted_engine::870::HostedEngine::(_handle_off) Engine down, local host does not have best score >MainThread::INFO::2013-11-14 15:28:34,536::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:28:34,784::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:28:34,861::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:28:34,920::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:28:34,921::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:28:34,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:28:34,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:28:34,922::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:28:34,922::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:28:34,922::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.215 >MainThread::INFO::2013-11-14 15:28:34,922::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:28:34,923::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435714|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435714 (Thu Nov 14 15:28:34 2013) >host-id=1 >score=0 >bridge=True >cpu-load=1.215 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:28:34,923::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:28:34,931::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:28:34,932::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:28:34,932::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435714, 'last-update-local-ts': 1384435714.93186, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:28:34,932::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:28:34,932::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435713, 'last-update-local-ts': 1384435714.93186, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:28:34,933::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:28:45,013::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:28:45,301::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:28:45,389::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:28:45,450::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:28:45,450::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:28:45,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:28:45,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:28:45,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:28:45,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:28:45,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.03 >MainThread::INFO::2013-11-14 15:28:45,453::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:28:45,453::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435725|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435725 (Thu Nov 14 15:28:45 2013) >host-id=1 >score=0 >bridge=True >cpu-load=1.03 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:28:45,453::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:28:45,463::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:28:45,463::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:28:45,464::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435725, 'last-update-local-ts': 1384435725.463068, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:28:45,464::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:28:45,464::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435724, 'last-update-local-ts': 1384435725.463068, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:28:45,465::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:28:55,556::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:28:55,814::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:28:55,900::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:28:55,961::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:28:55,961::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:28:55,961::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:28:55,961::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:28:55,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:28:55,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:28:55,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.87 >MainThread::INFO::2013-11-14 15:28:55,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:28:55,964::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435735|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435735 (Thu Nov 14 15:28:55 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.87 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:28:55,964::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:28:55,972::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:28:55,973::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:28:55,973::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435735, 'last-update-local-ts': 1384435735.9728191, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:28:55,973::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:28:55,973::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435734, 'last-update-local-ts': 1384435735.9728191, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:28:55,974::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:29:06,074::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:29:06,327::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:29:06,409::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:29:06,476::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:29:06,477::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:29:06,477::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:29:06,477::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:06,478::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:29:06,478::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:29:06,478::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.735 >MainThread::INFO::2013-11-14 15:29:06,478::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:06,479::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435746|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435746 (Thu Nov 14 15:29:06 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.735 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:29:06,479::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:29:06,487::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:29:06,488::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:29:06,488::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435746, 'last-update-local-ts': 1384435746.48786, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:06,488::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:29:06,488::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435745, 'last-update-local-ts': 1384435746.48786, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:06,489::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:29:16,575::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:29:16,837::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:29:16,921::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:29:16,981::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:29:16,981::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:29:16,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:29:16,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:16,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:29:16,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:29:16,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.665 >MainThread::INFO::2013-11-14 15:29:16,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:16,990::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435756|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435756 (Thu Nov 14 15:29:16 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.665 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:29:16,990::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:29:17,000::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:29:17,000::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:29:17,000::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435756, 'last-update-local-ts': 1384435757.0002849, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:17,001::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:29:17,001::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435755, 'last-update-local-ts': 1384435757.0002849, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:17,001::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:29:27,081::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:29:27,331::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:29:27,381::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:29:27,440::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:29:27,441::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:29:27,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:29:27,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:27,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:29:27,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:29:27,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.56 >MainThread::INFO::2013-11-14 15:29:27,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:27,444::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435767|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435767 (Thu Nov 14 15:29:27 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.56 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:29:27,444::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:29:27,453::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:29:27,453::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:29:27,453::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435767, 'last-update-local-ts': 1384435767.453239, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:27,454::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:29:27,454::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435765, 'last-update-local-ts': 1384435767.453239, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:27,454::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:29:37,548::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:29:37,797::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:29:37,875::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:29:37,935::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:29:37,936::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:29:37,936::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:29:37,936::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:37,937::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:29:37,937::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:29:37,937::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.555 >MainThread::INFO::2013-11-14 15:29:37,938::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:37,938::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435777|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435777 (Thu Nov 14 15:29:37 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.555 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:29:37,938::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:29:37,947::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:29:37,947::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:29:37,948::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435777, 'last-update-local-ts': 1384435777.9473341, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:37,948::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:29:37,948::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435776, 'last-update-local-ts': 1384435777.9473341, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:37,948::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:29:48,041::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:29:48,297::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:29:48,384::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:29:48,444::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:29:48,445::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:29:48,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:29:48,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:48,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:29:48,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:29:48,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.395 >MainThread::INFO::2013-11-14 15:29:48,453::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:48,453::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435788|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435788 (Thu Nov 14 15:29:48 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.395 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:29:48,453::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:29:48,464::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:29:48,464::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:29:48,464::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435788, 'last-update-local-ts': 1384435788.4640379, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:48,464::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:29:48,465::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435786, 'last-update-local-ts': 1384435788.4640379, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:48,465::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:29:58,559::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:29:58,811::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:29:58,904::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:29:58,964::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:29:58,964::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:29:58,965::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:29:58,965::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:58,965::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 15:29:58,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:29:58,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 15:29:58,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:29:58,967::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435798|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435798 (Thu Nov 14 15:29:58 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:29:58,968::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:29:58,975::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:29:58,975::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:29:58,976::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435798, 'last-update-local-ts': 1384435798.9753311, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:58,976::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:29:58,976::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435797, 'last-update-local-ts': 1384435798.9753311, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:29:58,976::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:30:09,064::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:30:09,321::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:30:09,444::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:30:09,506::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:30:09,507::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:30:09,507::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:30:09,507::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:09,508::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 15:30:09,508::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:30:09,508::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 15:30:09,508::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:09,509::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435809|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435809 (Thu Nov 14 15:30:09 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:30:09,510::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:30:09,519::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:30:09,520::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:30:09,520::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435809, 'last-update-local-ts': 1384435809.519578, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:09,520::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:30:09,520::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435807, 'last-update-local-ts': 1384435809.519578, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:09,520::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:30:19,594::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:30:19,849::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:30:19,919::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:30:19,979::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:30:19,980::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:30:19,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:30:19,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:19,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:30:19,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:30:19,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 15:30:19,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:19,982::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435819|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435819 (Thu Nov 14 15:30:19 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:30:19,982::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:30:19,989::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:30:19,989::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:30:19,989::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435819, 'last-update-local-ts': 1384435819.9891491, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:19,989::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:30:19,990::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435818, 'last-update-local-ts': 1384435819.9891491, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:19,990::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:30:30,068::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:30:30,315::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:30:30,409::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:30:30,471::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:30:30,472::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:30:30,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:30:30,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:30,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:30:30,481::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:30:30,481::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 15:30:30,481::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:30,482::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435830|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435830 (Thu Nov 14 15:30:30 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:30:30,482::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:30:30,493::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:30:30,493::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:30:30,493::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435830, 'last-update-local-ts': 1384435830.493207, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:30,494::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:30:30,494::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435828, 'last-update-local-ts': 1384435830.493207, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:30,494::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:30:40,576::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:30:40,830::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:30:40,939::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:30:41,014::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:30:41,014::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:30:41,015::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:30:41,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:41,020::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:30:41,023::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:30:41,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 15:30:41,026::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:41,027::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435841|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435841 (Thu Nov 14 15:30:41 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:30:41,027::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:30:41,037::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:30:41,038::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:30:41,038::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435841, 'last-update-local-ts': 1384435841.037307, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:41,038::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:30:41,038::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435839, 'last-update-local-ts': 1384435841.037307, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:41,039::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:30:51,116::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:30:51,369::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:30:51,492::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:30:51,567::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:30:51,567::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:30:51,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:30:51,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:51,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:30:51,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:30:51,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 15:30:51,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:30:51,576::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435851|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435851 (Thu Nov 14 15:30:51 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:30:51,576::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:30:51,583::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:30:51,583::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:30:51,583::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435851, 'last-update-local-ts': 1384435851.582947, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:51,583::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:30:51,583::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435849, 'last-update-local-ts': 1384435851.582947, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:30:51,584::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:31:01,656::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:31:01,912::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:31:01,981::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:31:02,041::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:31:02,041::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:31:02,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:31:02,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:02,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:31:02,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:31:02,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:31:02,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:02,044::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435862|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435862 (Thu Nov 14 15:31:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:31:02,044::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:31:02,053::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:31:02,053::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:31:02,053::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435862, 'last-update-local-ts': 1384435862.0531399, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:02,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:31:02,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435859, 'last-update-local-ts': 1384435862.0531399, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:02,054::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:31:12,151::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:31:12,462::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:31:12,559::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:31:12,628::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:31:12,628::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:31:12,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:31:12,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:12,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:31:12,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:31:12,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 15:31:12,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:12,630::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435872|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435872 (Thu Nov 14 15:31:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:31:12,631::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:31:12,637::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:31:12,638::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:31:12,638::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435872, 'last-update-local-ts': 1384435872.637857, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:12,638::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:31:12,638::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435870, 'last-update-local-ts': 1384435872.637857, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:12,639::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:31:22,716::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:31:22,966::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:31:23,061::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:31:23,123::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:31:23,123::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:31:23,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:31:23,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:23,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:31:23,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:31:23,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 15:31:23,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:23,132::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435883|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435883 (Thu Nov 14 15:31:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:31:23,132::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:31:23,141::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:31:23,142::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:31:23,142::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435883, 'last-update-local-ts': 1384435883.141628, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:23,142::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:31:23,142::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435880, 'last-update-local-ts': 1384435883.141628, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:23,142::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:31:33,224::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:31:33,476::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:31:33,569::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:31:33,629::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:31:33,630::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:31:33,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:31:33,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:33,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:31:33,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:31:33,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 15:31:33,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:33,633::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435893|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435893 (Thu Nov 14 15:31:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:31:33,633::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:31:33,641::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:31:33,641::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:31:33,642::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435893, 'last-update-local-ts': 1384435893.6413851, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:33,642::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:31:33,642::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435891, 'last-update-local-ts': 1384435893.6413851, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:33,642::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:31:43,721::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:31:43,976::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:31:44,059::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:31:44,120::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:31:44,120::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:31:44,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:31:44,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:44,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:31:44,122::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:31:44,122::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 15:31:44,122::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:44,122::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435904|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435904 (Thu Nov 14 15:31:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:31:44,123::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:31:44,131::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:31:44,131::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:31:44,131::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435904, 'last-update-local-ts': 1384435904.1313241, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:44,132::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:31:44,132::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435901, 'last-update-local-ts': 1384435904.1313241, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:44,132::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:31:54,205::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:31:54,459::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:31:54,528::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:31:54,587::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:31:54,587::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:31:54,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:31:54,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:54,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:31:54,595::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:31:54,595::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 15:31:54,595::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:31:54,596::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435914|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435914 (Thu Nov 14 15:31:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:31:54,596::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:31:54,606::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:31:54,606::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:31:54,606::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435914, 'last-update-local-ts': 1384435914.606293, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:54,607::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:31:54,607::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435912, 'last-update-local-ts': 1384435914.606293, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:31:54,607::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:32:04,702::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:32:04,952::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:32:05,037::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:32:05,097::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:32:05,098::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:32:05,098::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:32:05,098::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:05,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:32:05,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:32:05,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 15:32:05,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:05,101::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435925|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435925 (Thu Nov 14 15:32:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:32:05,101::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:32:05,109::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:32:05,110::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:32:05,110::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435925, 'last-update-local-ts': 1384435925.1095531, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:05,110::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:32:05,110::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435922, 'last-update-local-ts': 1384435925.1095531, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:05,110::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:32:15,191::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:32:15,446::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:32:15,544::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:32:15,605::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:32:15,605::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:32:15,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:32:15,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:15,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:32:15,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:32:15,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 15:32:15,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:15,614::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435935|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435935 (Thu Nov 14 15:32:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:32:15,614::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:32:15,624::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:32:15,624::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:32:15,624::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435935, 'last-update-local-ts': 1384435935.624208, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:15,625::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:32:15,625::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435933, 'last-update-local-ts': 1384435935.624208, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:15,625::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:32:25,714::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:32:25,971::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:32:26,050::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:32:26,112::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:32:26,112::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:32:26,113::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:32:26,113::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:26,113::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:32:26,113::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:32:26,114::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 15:32:26,114::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:26,115::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435946|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435946 (Thu Nov 14 15:32:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:32:26,115::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:32:26,124::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:32:26,124::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:32:26,125::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435946, 'last-update-local-ts': 1384435946.124476, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:26,125::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:32:26,125::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435943, 'last-update-local-ts': 1384435946.124476, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:26,125::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:32:36,205::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:32:36,451::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:32:36,515::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:32:36,575::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:32:36,575::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:32:36,576::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:32:36,576::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:36,576::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:32:36,577::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:32:36,577::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 15:32:36,577::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:36,578::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435956|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435956 (Thu Nov 14 15:32:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:32:36,578::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:32:36,585::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:32:36,585::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:32:36,586::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435956, 'last-update-local-ts': 1384435956.5853989, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:36,586::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:32:36,586::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435953, 'last-update-local-ts': 1384435956.5853989, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:36,586::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:32:46,663::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:32:46,969::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:32:47,070::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:32:47,133::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:32:47,133::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:32:47,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:32:47,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:47,142::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:32:47,142::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:32:47,142::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 15:32:47,143::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:47,143::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435967|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435967 (Thu Nov 14 15:32:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:32:47,143::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:32:47,157::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:32:47,158::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:32:47,158::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435967, 'last-update-local-ts': 1384435967.157604, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:47,158::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:32:47,158::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435964, 'last-update-local-ts': 1384435967.157604, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:47,159::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:32:57,225::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:32:57,479::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:32:57,538::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:32:57,598::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:32:57,598::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:32:57,599::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:32:57,599::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:57,599::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:32:57,599::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:32:57,600::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 15:32:57,600::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:32:57,601::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435977|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435977 (Thu Nov 14 15:32:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:32:57,601::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:32:57,610::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:32:57,610::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:32:57,611::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435977, 'last-update-local-ts': 1384435977.610368, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:57,611::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:32:57,611::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435974, 'last-update-local-ts': 1384435977.610368, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:32:57,611::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:33:07,706::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:33:07,973::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:33:08,064::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:33:08,126::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:33:08,126::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:33:08,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:33:08,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:08,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:33:08,128::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:33:08,128::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 15:33:08,128::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:08,129::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435988|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435988 (Thu Nov 14 15:33:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:33:08,129::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:33:08,137::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:33:08,138::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:33:08,138::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435988, 'last-update-local-ts': 1384435988.1377189, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:08,138::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:33:08,138::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435985, 'last-update-local-ts': 1384435988.1377189, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:08,139::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:33:18,222::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:33:18,479::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:33:18,571::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:33:18,633::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:33:18,633::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:33:18,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:33:18,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:18,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:33:18,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:33:18,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 15:33:18,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:18,641::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384435998|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384435998 (Thu Nov 14 15:33:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:33:18,641::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:33:18,652::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:33:18,652::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:33:18,652::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384435998, 'last-update-local-ts': 1384435998.6520219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:18,652::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:33:18,653::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384435995, 'last-update-local-ts': 1384435998.6520219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:18,653::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:33:28,739::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:33:29,064::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:33:29,143::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:33:29,205::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:33:29,205::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:33:29,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:33:29,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:29,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:33:29,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:33:29,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:33:29,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:29,208::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436009|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436009 (Thu Nov 14 15:33:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:33:29,208::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:33:29,218::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:33:29,219::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:33:29,219::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436009, 'last-update-local-ts': 1384436009.218802, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:29,219::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:33:29,219::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436006, 'last-update-local-ts': 1384436009.218802, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:29,220::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:33:39,297::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:33:39,579::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:33:39,641::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:33:39,701::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:33:39,702::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:33:39,702::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:33:39,702::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:39,703::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:33:39,703::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:33:39,703::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 15:33:39,703::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:39,704::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436019|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436019 (Thu Nov 14 15:33:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:33:39,704::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:33:39,712::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:33:39,713::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:33:39,713::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436019, 'last-update-local-ts': 1384436019.712652, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:39,713::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:33:39,713::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436016, 'last-update-local-ts': 1384436019.712652, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:39,713::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:33:49,799::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:33:50,050::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:33:50,131::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:33:50,191::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:33:50,191::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:33:50,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:33:50,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:50,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:33:50,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:33:50,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 15:33:50,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:33:50,200::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436030|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436030 (Thu Nov 14 15:33:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:33:50,200::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:33:50,210::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:33:50,211::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:33:50,211::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436030, 'last-update-local-ts': 1384436030.210659, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:50,211::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:33:50,211::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436027, 'last-update-local-ts': 1384436030.210659, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:33:50,212::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:34:00,297::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:34:00,550::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:34:00,645::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:34:00,705::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:34:00,705::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:34:00,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:34:00,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:00,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:34:00,707::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:34:00,707::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 15:34:00,707::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:00,708::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436040|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436040 (Thu Nov 14 15:34:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:34:00,709::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:34:00,718::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:34:00,718::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:34:00,718::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436040, 'last-update-local-ts': 1384436040.7180991, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:00,718::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:34:00,719::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436037, 'last-update-local-ts': 1384436040.7180991, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:00,719::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:34:10,795::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:34:11,069::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:34:11,187::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:34:11,254::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:34:11,254::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:34:11,261::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:34:11,261::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:11,262::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7035 >MainThread::INFO::2013-11-14 15:34:11,262::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:34:11,262::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 15:34:11,263::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:11,263::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436051|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436051 (Thu Nov 14 15:34:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7035 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:34:11,263::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:34:11,271::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:34:11,272::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:34:11,272::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436051, 'last-update-local-ts': 1384436051.2716379, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:11,272::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:34:11,272::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436047, 'last-update-local-ts': 1384436051.2716379, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:11,273::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:34:21,411::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:34:21,667::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:34:21,743::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:34:21,811::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:34:21,811::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:34:21,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:34:21,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:21,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:34:21,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:34:21,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 15:34:21,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:21,815::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436061|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436061 (Thu Nov 14 15:34:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:34:21,815::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:34:21,823::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:34:21,823::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:34:21,824::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436061, 'last-update-local-ts': 1384436061.823277, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:21,824::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:34:21,824::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436058, 'last-update-local-ts': 1384436061.823277, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:21,824::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:34:31,900::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:34:32,161::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:34:32,217::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:34:32,277::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:34:32,277::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:34:32,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:34:32,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:32,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:34:32,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:34:32,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 15:34:32,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:32,279::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436072|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436072 (Thu Nov 14 15:34:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:34:32,280::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:34:32,288::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:34:32,288::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:34:32,289::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436072, 'last-update-local-ts': 1384436072.288512, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:32,289::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:34:32,289::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436068, 'last-update-local-ts': 1384436072.288512, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:32,289::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:34:42,370::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:34:42,617::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:34:42,711::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:34:42,771::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:34:42,772::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:34:42,778::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:34:42,778::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:42,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:34:42,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:34:42,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 15:34:42,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:42,780::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436082|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436082 (Thu Nov 14 15:34:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:34:42,780::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:34:42,789::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:34:42,790::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:34:42,790::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436082, 'last-update-local-ts': 1384436082.7895679, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:42,790::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:34:42,790::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436079, 'last-update-local-ts': 1384436082.7895679, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:42,790::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:34:52,877::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:34:53,131::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:34:53,219::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:34:53,278::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:34:53,279::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:34:53,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:34:53,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:53,280::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:34:53,280::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:34:53,280::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 15:34:53,280::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:34:53,282::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436093|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436093 (Thu Nov 14 15:34:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:34:53,282::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:34:53,290::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:34:53,290::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:34:53,290::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436093, 'last-update-local-ts': 1384436093.2903211, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:53,291::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:34:53,291::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436089, 'last-update-local-ts': 1384436093.2903211, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:34:53,291::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:35:03,367::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:35:03,626::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:35:03,712::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:35:03,778::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:35:03,778::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:35:03,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:35:03,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:03,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:35:03,780::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:35:03,780::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 15:35:03,780::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:03,781::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436103|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436103 (Thu Nov 14 15:35:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:35:03,781::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:35:03,789::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:35:03,790::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:35:03,790::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436103, 'last-update-local-ts': 1384436103.7896121, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:03,790::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:35:03,790::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436100, 'last-update-local-ts': 1384436103.7896121, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:03,790::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:35:13,871::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:35:14,126::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:35:14,223::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:35:14,286::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:35:14,286::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:35:14,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:35:14,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:14,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:35:14,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:35:14,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 15:35:14,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:14,289::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436114|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436114 (Thu Nov 14 15:35:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:35:14,289::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:35:14,298::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:35:14,299::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:35:14,299::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436114, 'last-update-local-ts': 1384436114.2987521, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:14,299::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:35:14,299::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436110, 'last-update-local-ts': 1384436114.2987521, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:14,300::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:35:24,375::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:35:24,630::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:35:24,696::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:35:24,755::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:35:24,756::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:35:24,756::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:35:24,756::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:24,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:35:24,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:35:24,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 15:35:24,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:24,758::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436124|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436124 (Thu Nov 14 15:35:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:35:24,758::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:35:24,767::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:35:24,767::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:35:24,767::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436124, 'last-update-local-ts': 1384436124.767025, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:24,767::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:35:24,768::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436121, 'last-update-local-ts': 1384436124.767025, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:24,768::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:35:34,854::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:35:35,104::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:35:35,186::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:35:35,247::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:35:35,247::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:35:35,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:35:35,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:35,254::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:35:35,255::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:35:35,255::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 15:35:35,255::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:35,256::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436135|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436135 (Thu Nov 14 15:35:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:35:35,256::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:35:35,265::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:35:35,266::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:35:35,266::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436135, 'last-update-local-ts': 1384436135.265744, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:35,266::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:35:35,266::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436131, 'last-update-local-ts': 1384436135.265744, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:35,267::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:35:45,345::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:35:45,597::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:35:45,681::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:35:45,742::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:35:45,743::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:35:45,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:35:45,743::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:45,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:35:45,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:35:45,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 15:35:45,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:45,746::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436145|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436145 (Thu Nov 14 15:35:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:35:45,746::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:35:45,754::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:35:45,755::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:35:45,755::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436145, 'last-update-local-ts': 1384436145.754678, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:45,755::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:35:45,755::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436141, 'last-update-local-ts': 1384436145.754678, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:45,755::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:35:55,830::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:35:56,099::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:35:56,183::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:35:56,243::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:35:56,243::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:35:56,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:35:56,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:56,244::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:35:56,245::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:35:56,245::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 15:35:56,245::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:35:56,246::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436156|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436156 (Thu Nov 14 15:35:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:35:56,246::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:35:56,254::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:35:56,254::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:35:56,255::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436156, 'last-update-local-ts': 1384436156.254544, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:56,255::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:35:56,255::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436152, 'last-update-local-ts': 1384436156.254544, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:35:56,255::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:36:06,324::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:36:06,576::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:36:06,640::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:36:06,719::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:36:06,719::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:36:06,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:36:06,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:06,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 15:36:06,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:36:06,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 15:36:06,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:06,729::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436166|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436166 (Thu Nov 14 15:36:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:36:06,729::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:36:06,743::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:36:06,743::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:36:06,743::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436166, 'last-update-local-ts': 1384436166.7429199, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:06,743::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:36:06,744::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436162, 'last-update-local-ts': 1384436166.7429199, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:06,744::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:36:16,855::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:36:17,133::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:36:17,212::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:36:17,284::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:36:17,284::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:36:17,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:36:17,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:17,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:36:17,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:36:17,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 15:36:17,287::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:17,287::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436177|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436177 (Thu Nov 14 15:36:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:36:17,287::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:36:17,299::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:36:17,299::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:36:17,300::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436177, 'last-update-local-ts': 1384436177.2993469, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:17,300::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:36:17,300::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436173, 'last-update-local-ts': 1384436177.2993469, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:17,300::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:36:27,374::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:36:27,620::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:36:27,696::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:36:27,756::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:36:27,757::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:36:27,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:36:27,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:27,758::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:36:27,758::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:36:27,758::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 15:36:27,758::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:27,759::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436187|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436187 (Thu Nov 14 15:36:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:36:27,759::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:36:27,766::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:36:27,767::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:36:27,767::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436187, 'last-update-local-ts': 1384436187.7667551, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:27,767::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:36:27,767::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436183, 'last-update-local-ts': 1384436187.7667551, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:27,768::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:36:37,846::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:36:38,093::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:36:38,168::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:36:38,227::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:36:38,227::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:36:38,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:36:38,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:38,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:36:38,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:36:38,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 15:36:38,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:38,230::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436198|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436198 (Thu Nov 14 15:36:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:36:38,230::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:36:38,238::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:36:38,239::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:36:38,239::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436198, 'last-update-local-ts': 1384436198.238662, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:38,239::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:36:38,239::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436194, 'last-update-local-ts': 1384436198.238662, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:38,240::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:36:48,326::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:36:48,581::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:36:48,687::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:36:48,746::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:36:48,747::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:36:48,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:36:48,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:48,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:36:48,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:36:48,748::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 15:36:48,749::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:48,750::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436208|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436208 (Thu Nov 14 15:36:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:36:48,750::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:36:48,759::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:36:48,759::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:36:48,760::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436208, 'last-update-local-ts': 1384436208.759397, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:48,760::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:36:48,760::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436204, 'last-update-local-ts': 1384436208.759397, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:48,760::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:36:58,836::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:36:59,101::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:36:59,182::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:36:59,249::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:36:59,249::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:36:59,249::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:36:59,250::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:59,250::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:36:59,250::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:36:59,251::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 15:36:59,251::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:36:59,251::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436219|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436219 (Thu Nov 14 15:36:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:36:59,251::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:36:59,268::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:36:59,268::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:36:59,268::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436219, 'last-update-local-ts': 1384436219.267962, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:59,268::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:36:59,268::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436215, 'last-update-local-ts': 1384436219.267962, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:36:59,269::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:37:09,339::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:37:09,600::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:37:09,658::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:37:09,717::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:37:09,718::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:37:09,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:37:09,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:09,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:37:09,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:37:09,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 15:37:09,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:09,721::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436229|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436229 (Thu Nov 14 15:37:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:37:09,721::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:37:09,728::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:37:09,728::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:37:09,729::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436229, 'last-update-local-ts': 1384436229.7282519, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:09,729::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:37:09,729::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436225, 'last-update-local-ts': 1384436229.7282519, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:09,729::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:37:19,825::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:37:20,074::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:37:20,165::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:37:20,226::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:37:20,227::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:37:20,227::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:37:20,227::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:20,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:37:20,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:37:20,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 15:37:20,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:20,229::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436240|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436240 (Thu Nov 14 15:37:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:37:20,229::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:37:20,238::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:37:20,238::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:37:20,238::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436240, 'last-update-local-ts': 1384436240.2380509, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:20,238::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:37:20,239::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436236, 'last-update-local-ts': 1384436240.2380509, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:20,239::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:37:30,319::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:37:30,578::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:37:30,651::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:37:30,713::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:37:30,713::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:37:30,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:37:30,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:30,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:37:30,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:37:30,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 15:37:30,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:30,722::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436250|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436250 (Thu Nov 14 15:37:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:37:30,722::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:37:30,730::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:37:30,731::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:37:30,731::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436250, 'last-update-local-ts': 1384436250.7305491, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:30,731::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:37:30,731::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436246, 'last-update-local-ts': 1384436250.7305491, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:30,731::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:37:40,817::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:37:41,071::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:37:41,178::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:37:41,247::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:37:41,247::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:37:41,247::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:37:41,248::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:41,248::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 15:37:41,248::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:37:41,249::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 15:37:41,249::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:41,250::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436261|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436261 (Thu Nov 14 15:37:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:37:41,250::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:37:41,258::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:37:41,258::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:37:41,258::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436261, 'last-update-local-ts': 1384436261.2582021, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:41,259::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:37:41,259::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436256, 'last-update-local-ts': 1384436261.2582021, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:41,259::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:37:51,365::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:37:51,741::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:37:51,827::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:37:51,889::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:37:51,889::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:37:51,890::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:37:51,890::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:51,890::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:37:51,890::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:37:51,891::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:37:51,891::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:37:51,891::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436271|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436271 (Thu Nov 14 15:37:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:37:51,892::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:37:51,898::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:37:51,899::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:37:51,899::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436271, 'last-update-local-ts': 1384436271.898561, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:51,899::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:37:51,899::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436267, 'last-update-local-ts': 1384436271.898561, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:37:51,899::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:38:01,980::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:38:02,243::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:38:02,297::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:38:02,357::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:38:02,358::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:38:02,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:38:02,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:02,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:38:02,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:38:02,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 15:38:02,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:02,360::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436282|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436282 (Thu Nov 14 15:38:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:38:02,360::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:38:02,368::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:38:02,368::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:38:02,368::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436282, 'last-update-local-ts': 1384436282.3680871, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:02,368::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:38:02,369::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436277, 'last-update-local-ts': 1384436282.3680871, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:02,369::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:38:12,457::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:38:12,704::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:38:12,807::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:38:12,867::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:38:12,868::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:38:12,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:38:12,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:12,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:38:12,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:38:12,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 15:38:12,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:12,870::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436292|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436292 (Thu Nov 14 15:38:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:38:12,871::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:38:12,878::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:38:12,878::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:38:12,879::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436292, 'last-update-local-ts': 1384436292.8785329, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:12,879::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:38:12,879::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436288, 'last-update-local-ts': 1384436292.8785329, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:12,879::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:38:22,956::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:38:23,205::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:38:23,298::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:38:23,371::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:38:23,371::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:38:23,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:38:23,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:23,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:38:23,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:38:23,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 15:38:23,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:23,373::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436303|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436303 (Thu Nov 14 15:38:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:38:23,374::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:38:23,380::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:38:23,380::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:38:23,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436303, 'last-update-local-ts': 1384436303.380528, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:23,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:38:23,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436298, 'last-update-local-ts': 1384436303.380528, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:23,381::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:38:33,463::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:38:33,717::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:38:33,803::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:38:33,862::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:38:33,862::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:38:33,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:38:33,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:33,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:38:33,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:38:33,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 15:38:33,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:33,866::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436313|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436313 (Thu Nov 14 15:38:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:38:33,866::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:38:33,874::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:38:33,875::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:38:33,875::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436313, 'last-update-local-ts': 1384436313.874671, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:33,875::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:38:33,875::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436309, 'last-update-local-ts': 1384436313.874671, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:33,876::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:38:43,954::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:38:44,210::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:38:44,293::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:38:44,353::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:38:44,353::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:38:44,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:38:44,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:44,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:38:44,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:38:44,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 15:38:44,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:44,355::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436324|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436324 (Thu Nov 14 15:38:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:38:44,356::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:38:44,363::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:38:44,363::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:38:44,363::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436324, 'last-update-local-ts': 1384436324.3629341, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:44,363::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:38:44,363::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436319, 'last-update-local-ts': 1384436324.3629341, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:44,364::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:38:54,433::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:38:54,686::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:38:54,754::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:38:54,812::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:38:54,813::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:38:54,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:38:54,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:54,814::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:38:54,814::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:38:54,814::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 15:38:54,814::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:38:54,815::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436334|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436334 (Thu Nov 14 15:38:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:38:54,815::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:38:54,823::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:38:54,824::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:38:54,824::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436334, 'last-update-local-ts': 1384436334.8237841, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:54,824::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:38:54,824::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436330, 'last-update-local-ts': 1384436334.8237841, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:38:54,825::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:39:04,910::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:39:05,157::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:39:05,256::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:39:05,316::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:39:05,317::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:39:05,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:39:05,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:05,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:39:05,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:39:05,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 15:39:05,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:05,319::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436345|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436345 (Thu Nov 14 15:39:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:39:05,320::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:39:05,328::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:39:05,328::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:39:05,329::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436345, 'last-update-local-ts': 1384436345.3284559, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:05,329::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:39:05,329::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436340, 'last-update-local-ts': 1384436345.3284559, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:05,329::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:39:15,406::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:39:15,658::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:39:15,751::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:39:15,823::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:39:15,823::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:39:15,824::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:39:15,824::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:15,824::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:39:15,825::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:39:15,825::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 15:39:15,825::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:15,825::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436355|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436355 (Thu Nov 14 15:39:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:39:15,826::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:39:15,834::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:39:15,835::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:39:15,835::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436355, 'last-update-local-ts': 1384436355.834641, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:15,835::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:39:15,835::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436351, 'last-update-local-ts': 1384436355.834641, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:15,835::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:39:25,906::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:39:26,186::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:39:26,272::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:39:26,335::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:39:26,335::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:39:26,336::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:39:26,336::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:26,337::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:39:26,337::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:39:26,337::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 15:39:26,337::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:26,338::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436366|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436366 (Thu Nov 14 15:39:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:39:26,338::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:39:26,348::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:39:26,348::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:39:26,348::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436366, 'last-update-local-ts': 1384436366.3479991, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:26,348::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:39:26,349::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436361, 'last-update-local-ts': 1384436366.3479991, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:26,349::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:39:36,422::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:39:36,675::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:39:36,747::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:39:36,807::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:39:36,808::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:39:36,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:39:36,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:36,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:39:36,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:39:36,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 15:39:36,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:36,810::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436376|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436376 (Thu Nov 14 15:39:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:39:36,810::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:39:36,818::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:39:36,819::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:39:36,819::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436376, 'last-update-local-ts': 1384436376.818712, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:36,819::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:39:36,819::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436372, 'last-update-local-ts': 1384436376.818712, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:36,820::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:39:46,892::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:39:47,168::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:39:47,238::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:39:47,299::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:39:47,300::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:39:47,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:39:47,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:47,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:39:47,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:39:47,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 15:39:47,308::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:47,308::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436387|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436387 (Thu Nov 14 15:39:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:39:47,308::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:39:47,318::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:39:47,318::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:39:47,319::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436387, 'last-update-local-ts': 1384436387.3184519, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:47,319::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:39:47,319::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436382, 'last-update-local-ts': 1384436387.3184519, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:47,319::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:39:57,399::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:39:57,646::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:39:57,723::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:39:57,782::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:39:57,783::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:39:57,783::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:39:57,783::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:57,784::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:39:57,784::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:39:57,784::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 15:39:57,784::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:39:57,785::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436397|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436397 (Thu Nov 14 15:39:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:39:57,786::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:39:57,794::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:39:57,794::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:39:57,795::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436397, 'last-update-local-ts': 1384436397.7945371, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:57,795::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:39:57,795::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436392, 'last-update-local-ts': 1384436397.7945371, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:39:57,796::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:40:07,875::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:40:08,136::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:40:08,282::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:40:08,350::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:40:08,350::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:40:08,356::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:40:08,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:08,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:40:08,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:40:08,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:40:08,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:08,359::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436408|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436408 (Thu Nov 14 15:40:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:40:08,359::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:40:08,369::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:40:08,370::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:40:08,370::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436408, 'last-update-local-ts': 1384436408.3696809, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:08,370::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:40:08,370::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436403, 'last-update-local-ts': 1384436408.3696809, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:08,371::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:40:18,439::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:40:18,704::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:40:18,801::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:40:18,867::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:40:18,868::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:40:18,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:40:18,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:18,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:40:18,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:40:18,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 15:40:18,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:18,870::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436418|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436418 (Thu Nov 14 15:40:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:40:18,870::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:40:18,879::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:40:18,880::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:40:18,880::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436418, 'last-update-local-ts': 1384436418.8798251, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:18,880::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:40:18,880::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436413, 'last-update-local-ts': 1384436418.8798251, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:18,881::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:40:28,948::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:40:29,202::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:40:29,268::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:40:29,343::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:40:29,344::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:40:29,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:40:29,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:29,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:40:29,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:40:29,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 15:40:29,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:29,352::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436429|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436429 (Thu Nov 14 15:40:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:40:29,353::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:40:29,362::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:40:29,363::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:40:29,363::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436429, 'last-update-local-ts': 1384436429.3625481, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:29,363::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:40:29,363::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436424, 'last-update-local-ts': 1384436429.3625481, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:29,363::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:40:39,453::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:40:39,700::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:40:39,798::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:40:39,858::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:40:39,859::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:40:39,859::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:40:39,859::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:39,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:40:39,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:40:39,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 15:40:39,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:39,861::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436439|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436439 (Thu Nov 14 15:40:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:40:39,862::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:40:39,869::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:40:39,870::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:40:39,870::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436439, 'last-update-local-ts': 1384436439.8697231, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:39,870::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:40:39,870::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436434, 'last-update-local-ts': 1384436439.8697231, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:39,871::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:40:49,946::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:40:50,207::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:40:50,293::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:40:50,365::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:40:50,365::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:40:50,366::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:40:50,366::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:50,366::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:40:50,367::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:40:50,367::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 15:40:50,367::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:40:50,368::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436450|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436450 (Thu Nov 14 15:40:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:40:50,368::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:40:50,376::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:40:50,377::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:40:50,377::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436450, 'last-update-local-ts': 1384436450.376627, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:50,377::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:40:50,377::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436445, 'last-update-local-ts': 1384436450.376627, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:40:50,377::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:41:00,449::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:41:00,702::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:41:00,792::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:41:00,852::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:41:00,852::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:41:00,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:41:00,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:00,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:41:00,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:41:00,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 15:41:00,854::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:00,855::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436460|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436460 (Thu Nov 14 15:41:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:41:00,855::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:41:00,865::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:41:00,865::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:41:00,866::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436460, 'last-update-local-ts': 1384436460.8653729, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:00,866::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:41:00,866::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436455, 'last-update-local-ts': 1384436460.8653729, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:00,866::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:41:10,938::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:41:11,186::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:41:11,247::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:41:11,306::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:41:11,307::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:41:11,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:41:11,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:11,308::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:41:11,308::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:41:11,308::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 15:41:11,308::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:11,309::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436471|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436471 (Thu Nov 14 15:41:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:41:11,309::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:41:11,320::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:41:11,321::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:41:11,321::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436471, 'last-update-local-ts': 1384436471.320622, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:11,321::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:41:11,322::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436465, 'last-update-local-ts': 1384436471.320622, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:11,322::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:41:21,396::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:41:21,698::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:41:21,864::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:41:21,928::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:41:21,928::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:41:21,928::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:41:21,929::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:21,929::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:41:21,929::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:41:21,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 15:41:21,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:21,930::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436481|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436481 (Thu Nov 14 15:41:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:41:21,931::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:41:21,945::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:41:21,946::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:41:21,946::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436481, 'last-update-local-ts': 1384436481.9457459, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:21,946::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:41:21,946::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436476, 'last-update-local-ts': 1384436481.9457459, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:21,947::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:41:32,033::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:41:32,285::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:41:32,378::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:41:32,440::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:41:32,440::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:41:32,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:41:32,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:32,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:41:32,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:41:32,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.34 >MainThread::INFO::2013-11-14 15:41:32,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:32,443::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436492|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436492 (Thu Nov 14 15:41:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.34 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:41:32,443::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:41:32,452::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:41:32,452::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:41:32,452::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436492, 'last-update-local-ts': 1384436492.4519939, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:32,452::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:41:32,453::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436486, 'last-update-local-ts': 1384436492.4519939, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:32,453::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:41:42,530::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:41:42,784::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:41:42,872::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:41:42,933::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:41:42,933::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:41:42,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:41:42,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:42,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:41:42,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:41:42,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 15:41:42,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:42,936::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436502|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436502 (Thu Nov 14 15:41:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.32 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:41:42,936::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:41:42,944::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:41:42,945::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:41:42,945::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436502, 'last-update-local-ts': 1384436502.9446499, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:42,945::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:41:42,945::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436497, 'last-update-local-ts': 1384436502.9446499, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:42,945::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:41:53,018::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:41:53,271::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:41:53,338::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:41:53,450::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:41:53,450::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:41:53,450::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:41:53,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:53,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:41:53,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:41:53,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 15:41:53,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:41:53,453::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436513|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436513 (Thu Nov 14 15:41:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:41:53,453::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:41:53,462::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:41:53,462::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:41:53,462::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436513, 'last-update-local-ts': 1384436513.462177, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:53,463::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:41:53,463::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436507, 'last-update-local-ts': 1384436513.462177, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:41:53,463::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:42:03,581::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:42:03,876::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:42:03,968::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:42:04,038::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:42:04,038::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:42:04,039::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:42:04,039::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:04,039::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:42:04,040::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:42:04,040::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.74 >MainThread::INFO::2013-11-14 15:42:04,040::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:04,041::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436524|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436524 (Thu Nov 14 15:42:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.74 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:42:04,041::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:42:04,054::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:42:04,055::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:42:04,055::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436524, 'last-update-local-ts': 1384436524.0548091, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:04,055::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:42:04,055::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436518, 'last-update-local-ts': 1384436524.0548091, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:04,056::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:42:14,127::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:42:14,385::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:42:14,466::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:42:14,527::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:42:14,527::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:42:14,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:42:14,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:14,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:42:14,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:42:14,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.665 >MainThread::INFO::2013-11-14 15:42:14,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:14,529::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436534|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436534 (Thu Nov 14 15:42:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.665 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:42:14,530::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:42:14,538::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:42:14,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:42:14,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436534, 'last-update-local-ts': 1384436534.538224, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:14,539::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:42:14,539::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436528, 'last-update-local-ts': 1384436534.538224, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:14,539::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:42:24,624::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:42:24,878::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:42:24,961::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:42:25,021::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:42:25,022::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:42:25,022::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:42:25,022::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:25,023::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:42:25,023::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:42:25,023::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.6 >MainThread::INFO::2013-11-14 15:42:25,024::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:25,024::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436545|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436545 (Thu Nov 14 15:42:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.6 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:42:25,024::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:42:25,033::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:42:25,033::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:42:25,033::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436545, 'last-update-local-ts': 1384436545.0330091, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:25,033::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:42:25,034::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436539, 'last-update-local-ts': 1384436545.0330091, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:25,034::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:42:35,137::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:42:35,389::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:42:35,471::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:42:35,539::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:42:35,539::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:42:35,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:42:35,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:35,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:42:35,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:42:35,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.54 >MainThread::INFO::2013-11-14 15:42:35,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:35,542::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436555|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436555 (Thu Nov 14 15:42:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.54 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:42:35,542::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:42:35,551::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:42:35,551::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:42:35,552::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436555, 'last-update-local-ts': 1384436555.5513871, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:35,552::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:42:35,552::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436549, 'last-update-local-ts': 1384436555.5513871, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:35,552::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:42:45,638::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:42:45,886::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:42:45,961::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:42:46,021::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:42:46,021::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:42:46,027::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:42:46,028::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:46,028::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:42:46,028::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:42:46,029::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.46 >MainThread::INFO::2013-11-14 15:42:46,029::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:46,029::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436566|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436566 (Thu Nov 14 15:42:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.46 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:42:46,029::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:42:46,040::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:42:46,040::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:42:46,040::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436566, 'last-update-local-ts': 1384436566.039897, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:46,040::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:42:46,040::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436560, 'last-update-local-ts': 1384436566.039897, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:46,041::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:42:56,116::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:42:56,391::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:42:56,463::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:42:56,525::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:42:56,525::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:42:56,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:42:56,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:56,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:42:56,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:42:56,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.39 >MainThread::INFO::2013-11-14 15:42:56,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:42:56,528::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436576|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436576 (Thu Nov 14 15:42:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.39 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:42:56,528::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:42:56,537::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:42:56,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:42:56,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436576, 'last-update-local-ts': 1384436576.537564, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:56,538::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:42:56,538::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436570, 'last-update-local-ts': 1384436576.537564, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:42:56,538::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:43:06,614::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:43:06,867::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:43:06,928::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:43:06,991::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:43:06,991::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:43:06,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:43:06,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:06,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:43:06,993::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:43:06,993::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.33 >MainThread::INFO::2013-11-14 15:43:06,993::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:06,994::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436586|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436586 (Thu Nov 14 15:43:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.33 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:43:06,994::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:43:07,002::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:43:07,003::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:43:07,003::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436586, 'last-update-local-ts': 1384436587.002676, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:07,003::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:43:07,003::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436580, 'last-update-local-ts': 1384436587.002676, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:07,003::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:43:17,088::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:43:17,340::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:43:17,429::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:43:17,489::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:43:17,489::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:43:17,496::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:43:17,496::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:17,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 15:43:17,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:43:17,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.355 >MainThread::INFO::2013-11-14 15:43:17,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:17,498::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436597|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436597 (Thu Nov 14 15:43:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.355 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:43:17,498::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:43:17,508::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:43:17,508::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:43:17,508::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436597, 'last-update-local-ts': 1384436597.508265, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:17,509::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:43:17,509::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436591, 'last-update-local-ts': 1384436597.508265, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:17,509::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:43:27,587::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:43:27,837::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:43:27,918::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:43:27,979::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:43:27,980::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:43:27,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:43:27,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:27,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 15:43:27,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:43:27,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 15:43:27,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:27,983::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436607|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436607 (Thu Nov 14 15:43:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:43:27,983::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:43:27,991::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:43:27,992::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:43:27,992::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436607, 'last-update-local-ts': 1384436607.991698, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:27,992::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:43:27,992::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436601, 'last-update-local-ts': 1384436607.991698, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:27,993::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:43:38,062::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:43:38,318::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:43:38,388::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:43:38,456::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:43:38,456::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:43:38,456::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:43:38,457::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:38,457::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7045 >MainThread::INFO::2013-11-14 15:43:38,457::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:43:38,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 15:43:38,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:38,458::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436618|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436618 (Thu Nov 14 15:43:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7045 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:43:38,458::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:43:38,467::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:43:38,467::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:43:38,467::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436618, 'last-update-local-ts': 1384436618.4670379, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:38,467::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:43:38,468::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436612, 'last-update-local-ts': 1384436618.4670379, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:38,468::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:43:48,541::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:43:48,800::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:43:48,857::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:43:48,919::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:43:48,919::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:43:48,920::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:43:48,920::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:48,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:43:48,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:43:48,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 15:43:48,922::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:48,922::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436628|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436628 (Thu Nov 14 15:43:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:43:48,923::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:43:48,933::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:43:48,934::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:43:48,934::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436628, 'last-update-local-ts': 1384436628.933583, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:48,934::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:43:48,934::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436622, 'last-update-local-ts': 1384436628.933583, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:48,935::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:43:59,032::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:43:59,279::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:43:59,371::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:43:59,430::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:43:59,431::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:43:59,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:43:59,431::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:59,432::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 15:43:59,432::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:43:59,432::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 15:43:59,432::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:43:59,433::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436639|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436639 (Thu Nov 14 15:43:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:43:59,434::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:43:59,442::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:43:59,443::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:43:59,443::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436639, 'last-update-local-ts': 1384436639.4427879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:59,443::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:43:59,443::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436633, 'last-update-local-ts': 1384436639.4427879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:43:59,444::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:44:09,510::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:44:09,763::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:44:09,883::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:44:09,945::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:44:09,945::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:44:09,951::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:44:09,952::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:09,952::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 15:44:09,952::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:44:09,953::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 15:44:09,953::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:09,953::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436649|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436649 (Thu Nov 14 15:44:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:44:09,954::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:44:09,963::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:44:09,964::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:44:09,964::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436649, 'last-update-local-ts': 1384436649.9637649, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:09,964::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:44:09,964::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436643, 'last-update-local-ts': 1384436649.9637649, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:09,965::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:44:20,050::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:44:20,307::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:44:20,405::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:44:20,466::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:44:20,466::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:44:20,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:44:20,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:20,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:44:20,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:44:20,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.49 >MainThread::INFO::2013-11-14 15:44:20,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:20,469::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436660|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436660 (Thu Nov 14 15:44:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.49 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:44:20,469::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:44:20,477::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:44:20,477::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:44:20,477::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436660, 'last-update-local-ts': 1384436660.477088, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:20,477::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:44:20,478::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436653, 'last-update-local-ts': 1384436660.477088, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:20,478::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:44:30,552::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:44:30,800::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:44:30,870::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:44:30,930::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:44:30,931::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:44:30,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:44:30,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:30,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:44:30,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:44:30,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.415 >MainThread::INFO::2013-11-14 15:44:30,933::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:30,933::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436670|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436670 (Thu Nov 14 15:44:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.415 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:44:30,933::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:44:30,941::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:44:30,942::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:44:30,942::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436670, 'last-update-local-ts': 1384436670.9417, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:30,942::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:44:30,942::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436664, 'last-update-local-ts': 1384436670.9417, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:30,943::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:44:41,028::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:44:41,279::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:44:41,357::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:44:41,417::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:44:41,417::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:44:41,425::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:44:41,425::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:41,426::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:44:41,426::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:44:41,426::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.35 >MainThread::INFO::2013-11-14 15:44:41,427::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:41,427::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436681|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436681 (Thu Nov 14 15:44:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.35 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:44:41,427::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:44:41,439::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:44:41,440::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:44:41,440::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436681, 'last-update-local-ts': 1384436681.4396479, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:41,440::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:44:41,440::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436674, 'last-update-local-ts': 1384436681.4396479, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:41,441::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:44:51,546::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:44:51,821::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:44:51,926::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:44:51,985::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:44:51,985::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:44:51,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:44:51,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:51,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 15:44:51,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:44:51,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.295 >MainThread::INFO::2013-11-14 15:44:51,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:44:51,988::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436691|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436691 (Thu Nov 14 15:44:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.295 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:44:51,989::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:44:51,995::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:44:51,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:44:51,996::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436691, 'last-update-local-ts': 1384436691.9956779, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:51,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:44:51,996::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436685, 'last-update-local-ts': 1384436691.9956779, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:44:51,997::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:45:02,073::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:45:02,330::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:45:02,428::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:45:02,489::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:45:02,489::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:45:02,496::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:45:02,496::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:02,496::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:45:02,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:45:02,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.29 >MainThread::INFO::2013-11-14 15:45:02,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:02,498::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436702|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436702 (Thu Nov 14 15:45:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.29 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:45:02,498::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:45:02,505::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:45:02,505::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:45:02,505::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436702, 'last-update-local-ts': 1384436702.505291, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:02,506::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:45:02,506::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436695, 'last-update-local-ts': 1384436702.505291, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:02,506::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:45:12,587::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:45:12,843::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:45:12,919::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:45:12,979::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:45:12,979::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:45:12,979::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:45:12,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:12,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:45:12,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:45:12,980::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.245 >MainThread::INFO::2013-11-14 15:45:12,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:12,982::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436712|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436712 (Thu Nov 14 15:45:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.245 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:45:12,982::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:45:12,991::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:45:12,991::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:45:12,992::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436712, 'last-update-local-ts': 1384436712.9913671, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:12,992::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:45:12,992::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436706, 'last-update-local-ts': 1384436712.9913671, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:12,993::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:45:23,068::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:45:23,336::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:45:23,396::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:45:23,457::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:45:23,457::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:45:23,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:45:23,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:23,458::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:45:23,459::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:45:23,459::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 15:45:23,459::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:23,459::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436723|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436723 (Thu Nov 14 15:45:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:45:23,460::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:45:23,468::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:45:23,468::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:45:23,469::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436723, 'last-update-local-ts': 1384436723.4684761, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:23,469::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:45:23,469::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436716, 'last-update-local-ts': 1384436723.4684761, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:23,469::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:45:33,554::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:45:33,807::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:45:33,895::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:45:33,955::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:45:33,956::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:45:33,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:45:33,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:33,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:45:33,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:45:33,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 15:45:33,964::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:33,964::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436733|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436733 (Thu Nov 14 15:45:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:45:33,964::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:45:33,973::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:45:33,973::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:45:33,973::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436733, 'last-update-local-ts': 1384436733.9729421, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:33,973::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:45:33,973::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436727, 'last-update-local-ts': 1384436733.9729421, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:33,974::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:45:44,060::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:45:44,310::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:45:44,391::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:45:44,451::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:45:44,451::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:45:44,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:45:44,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:44,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:45:44,453::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:45:44,453::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 15:45:44,453::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:44,455::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436744|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436744 (Thu Nov 14 15:45:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:45:44,455::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:45:44,464::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:45:44,464::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:45:44,464::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436744, 'last-update-local-ts': 1384436744.463999, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:44,464::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:45:44,465::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436737, 'last-update-local-ts': 1384436744.463999, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:44,465::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:45:54,538::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:45:54,787::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:45:54,845::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:45:54,905::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:45:54,905::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:45:54,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:45:54,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:54,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:45:54,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:45:54,907::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 15:45:54,907::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:45:54,907::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436754|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436754 (Thu Nov 14 15:45:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:45:54,908::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:45:54,914::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:45:54,915::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:45:54,915::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436754, 'last-update-local-ts': 1384436754.9147379, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:54,915::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:45:54,915::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436748, 'last-update-local-ts': 1384436754.9147379, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:45:54,916::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:46:04,997::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:46:05,250::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:46:05,348::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:46:05,408::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:46:05,408::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:46:05,415::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:46:05,415::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:05,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:46:05,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:46:05,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 15:46:05,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:05,417::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436765|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436765 (Thu Nov 14 15:46:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:46:05,417::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:46:05,427::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:46:05,428::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:46:05,428::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436765, 'last-update-local-ts': 1384436765.4278519, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:05,428::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:46:05,428::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436758, 'last-update-local-ts': 1384436765.4278519, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:05,429::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:46:15,523::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:46:15,774::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:46:15,870::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:46:15,929::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:46:15,929::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:46:15,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:46:15,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:15,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:46:15,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:46:15,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 15:46:15,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:15,932::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436775|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436775 (Thu Nov 14 15:46:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:46:15,933::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:46:15,941::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:46:15,942::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:46:15,942::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436775, 'last-update-local-ts': 1384436775.9416599, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:15,942::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:46:15,942::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436768, 'last-update-local-ts': 1384436775.9416599, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:15,942::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:46:26,010::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:46:26,277::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:46:26,388::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:46:26,458::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:46:26,459::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:46:26,465::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:46:26,466::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:26,466::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:46:26,466::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:46:26,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 15:46:26,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:26,467::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436786|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436786 (Thu Nov 14 15:46:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:46:26,467::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:46:26,477::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:46:26,477::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:46:26,477::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436786, 'last-update-local-ts': 1384436786.4769909, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:26,477::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:46:26,478::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436779, 'last-update-local-ts': 1384436786.4769909, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:26,478::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:46:36,556::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:46:36,810::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:46:36,889::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:46:36,948::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:46:36,948::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:46:36,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:46:36,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:36,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:46:36,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:46:36,950::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 15:46:36,950::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:36,951::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436796|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436796 (Thu Nov 14 15:46:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:46:36,951::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:46:36,960::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:46:36,961::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:46:36,961::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436796, 'last-update-local-ts': 1384436796.9607811, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:36,961::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:46:36,961::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436789, 'last-update-local-ts': 1384436796.9607811, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:36,962::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:46:47,028::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:46:47,286::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:46:47,352::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:46:47,413::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:46:47,413::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:46:47,414::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:46:47,414::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:47,414::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:46:47,415::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:46:47,415::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 15:46:47,415::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:47,415::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436807|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436807 (Thu Nov 14 15:46:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:46:47,416::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:46:47,424::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:46:47,424::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:46:47,425::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436807, 'last-update-local-ts': 1384436807.4245141, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:47,425::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:46:47,425::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436800, 'last-update-local-ts': 1384436807.4245141, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:47,425::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:46:57,511::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:46:57,759::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:46:57,838::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:46:57,898::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:46:57,898::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:46:57,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:46:57,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:57,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:46:57,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:46:57,900::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 15:46:57,900::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:46:57,900::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436817|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436817 (Thu Nov 14 15:46:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.265 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:46:57,901::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:46:57,911::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:46:57,911::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:46:57,911::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436817, 'last-update-local-ts': 1384436817.910918, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:57,911::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:46:57,911::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436810, 'last-update-local-ts': 1384436817.910918, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:46:57,912::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:47:07,996::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:47:08,248::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:47:08,345::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:47:08,405::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:47:08,406::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:47:08,406::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:47:08,406::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:08,407::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:47:08,407::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:47:08,407::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.26 >MainThread::INFO::2013-11-14 15:47:08,407::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:08,409::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436828|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436828 (Thu Nov 14 15:47:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.26 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:47:08,409::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:47:08,417::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:47:08,417::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:47:08,418::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436828, 'last-update-local-ts': 1384436828.4174931, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:08,418::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:47:08,418::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436821, 'last-update-local-ts': 1384436828.4174931, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:08,418::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:47:18,495::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:47:18,749::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:47:18,831::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:47:18,904::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:47:18,904::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:47:18,905::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:47:18,905::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:18,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:47:18,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:47:18,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 15:47:18,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:18,907::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436838|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436838 (Thu Nov 14 15:47:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:47:18,907::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:47:18,916::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:47:18,916::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:47:18,916::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436838, 'last-update-local-ts': 1384436838.916245, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:18,917::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:47:18,917::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436831, 'last-update-local-ts': 1384436838.916245, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:18,917::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:47:28,989::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:47:29,249::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:47:29,313::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:47:29,372::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:47:29,372::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:47:29,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:47:29,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:29,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:47:29,374::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:47:29,374::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 15:47:29,374::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:29,375::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436849|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436849 (Thu Nov 14 15:47:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:47:29,375::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:47:29,384::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:47:29,384::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:47:29,384::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436849, 'last-update-local-ts': 1384436849.383985, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:29,384::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:47:29,385::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436842, 'last-update-local-ts': 1384436849.383985, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:29,385::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:47:39,476::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:47:39,726::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:47:39,814::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:47:39,874::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:47:39,874::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:47:39,875::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:47:39,875::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:39,875::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:47:39,876::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:47:39,876::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 15:47:39,876::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:39,877::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436859|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436859 (Thu Nov 14 15:47:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:47:39,877::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:47:39,885::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:47:39,886::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:47:39,886::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436859, 'last-update-local-ts': 1384436859.885571, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:39,886::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:47:39,886::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436852, 'last-update-local-ts': 1384436859.885571, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:39,886::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:47:49,963::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:47:50,218::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:47:50,309::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:47:50,369::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:47:50,370::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:47:50,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:47:50,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:50,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:47:50,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:47:50,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 15:47:50,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:47:50,372::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436870|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436870 (Thu Nov 14 15:47:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:47:50,372::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:47:50,381::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:47:50,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:47:50,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436870, 'last-update-local-ts': 1384436870.380959, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:50,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:47:50,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436862, 'last-update-local-ts': 1384436870.380959, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:47:50,382::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:48:00,457::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:48:00,717::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:48:00,806::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:48:00,871::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:48:00,871::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:48:00,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:48:00,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:00,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:48:00,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:48:00,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 15:48:00,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:00,874::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436880|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436880 (Thu Nov 14 15:48:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:48:00,874::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:48:00,884::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:48:00,884::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:48:00,884::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436880, 'last-update-local-ts': 1384436880.8838971, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:00,884::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:48:00,884::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436873, 'last-update-local-ts': 1384436880.8838971, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:00,885::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:48:10,967::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:48:11,219::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:48:11,309::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:48:11,369::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:48:11,369::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:48:11,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:48:11,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:11,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:48:11,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:48:11,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 15:48:11,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:11,372::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436891|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436891 (Thu Nov 14 15:48:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:48:11,372::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:48:11,380::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:48:11,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:48:11,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436891, 'last-update-local-ts': 1384436891.3807559, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:11,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:48:11,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436883, 'last-update-local-ts': 1384436891.3807559, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:11,382::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:48:21,447::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:48:21,850::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:48:21,946::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:48:22,007::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:48:22,007::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:48:22,007::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:48:22,008::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:22,008::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7036 >MainThread::INFO::2013-11-14 15:48:22,008::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:48:22,009::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 15:48:22,009::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:22,010::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436902|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436902 (Thu Nov 14 15:48:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7036 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:48:22,010::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:48:22,017::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:48:22,017::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:48:22,017::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436902, 'last-update-local-ts': 1384436902.0169761, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:22,017::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:48:22,018::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436894, 'last-update-local-ts': 1384436902.0169761, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:22,018::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:48:32,112::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:48:32,371::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:48:32,421::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:48:32,481::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:48:32,481::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:48:32,482::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:48:32,482::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:32,482::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 15:48:32,482::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:48:32,483::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 15:48:32,483::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:32,483::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436912|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436912 (Thu Nov 14 15:48:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:48:32,484::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:48:32,491::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:48:32,491::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:48:32,491::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436912, 'last-update-local-ts': 1384436912.4910409, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:32,491::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:48:32,492::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436904, 'last-update-local-ts': 1384436912.4910409, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:32,492::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:48:42,574::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:48:42,845::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:48:42,937::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:48:42,997::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:48:42,997::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:48:42,998::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:48:42,998::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:42,998::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:48:42,999::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:48:42,999::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 15:48:42,999::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:42,999::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436922|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436922 (Thu Nov 14 15:48:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:48:43,000::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:48:43,006::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:48:43,007::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:48:43,007::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436922, 'last-update-local-ts': 1384436923.0066071, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:43,007::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:48:43,007::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436915, 'last-update-local-ts': 1384436923.0066071, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:43,007::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:48:53,088::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:48:53,343::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:48:53,431::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:48:53,491::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:48:53,491::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:48:53,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:48:53,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:53,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:48:53,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:48:53,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 15:48:53,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:48:53,494::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436933|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436933 (Thu Nov 14 15:48:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:48:53,494::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:48:53,503::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:48:53,503::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:48:53,503::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436933, 'last-update-local-ts': 1384436933.5031991, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:53,504::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:48:53,504::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436925, 'last-update-local-ts': 1384436933.5031991, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:48:53,504::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:49:03,607::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:49:03,862::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:49:03,941::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:49:04,009::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:49:04,009::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:49:04,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:49:04,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:04,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:49:04,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:49:04,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 15:49:04,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:04,011::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436944|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436944 (Thu Nov 14 15:49:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:49:04,012::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:49:04,020::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:49:04,020::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:49:04,021::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436944, 'last-update-local-ts': 1384436944.020359, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:04,021::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:49:04,021::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436946, 'last-update-local-ts': 1384436944.020359, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:04,021::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:49:14,100::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:49:14,358::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:49:14,420::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:49:14,481::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:49:14,481::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:49:14,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:49:14,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:14,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:49:14,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:49:14,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 15:49:14,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:14,490::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436954|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436954 (Thu Nov 14 15:49:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:49:14,490::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:49:14,498::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:49:14,499::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:49:14,499::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436954, 'last-update-local-ts': 1384436954.4988561, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:14,499::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:49:14,499::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436957, 'last-update-local-ts': 1384436954.4988561, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:14,500::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:49:24,575::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:49:24,831::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:49:24,933::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:49:24,999::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:49:24,999::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:49:25,000::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:49:25,000::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:25,000::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:49:25,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:49:25,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 15:49:25,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:25,002::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436965|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436965 (Thu Nov 14 15:49:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:49:25,002::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:49:25,012::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:49:25,012::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:49:25,012::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436965, 'last-update-local-ts': 1384436965.0120709, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:25,012::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:49:25,013::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436967, 'last-update-local-ts': 1384436965.0120709, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:25,013::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:49:35,106::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:49:35,361::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:49:35,446::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:49:35,507::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:49:35,508::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:49:35,508::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:49:35,508::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:35,509::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:49:35,509::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:49:35,509::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 15:49:35,510::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:35,510::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436975|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436975 (Thu Nov 14 15:49:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:49:35,510::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:49:35,519::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:49:35,519::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:49:35,519::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436975, 'last-update-local-ts': 1384436975.5191579, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:35,520::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:49:35,520::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436977, 'last-update-local-ts': 1384436975.5191579, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:35,520::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:49:45,607::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:49:45,862::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:49:45,955::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:49:46,016::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:49:46,016::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:49:46,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:49:46,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:46,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:49:46,018::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:49:46,018::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 15:49:46,018::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:46,019::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436986|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436986 (Thu Nov 14 15:49:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:49:46,019::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:49:46,028::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:49:46,028::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:49:46,028::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436986, 'last-update-local-ts': 1384436986.0282209, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:46,029::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:49:46,029::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436988, 'last-update-local-ts': 1384436986.0282209, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:46,029::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:49:56,120::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:49:56,370::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:49:56,447::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:49:56,514::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:49:56,515::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:49:56,515::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:49:56,515::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:56,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 15:49:56,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:49:56,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 15:49:56,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:49:56,517::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384436996|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384436996 (Thu Nov 14 15:49:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:49:56,517::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:49:56,528::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:49:56,529::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:49:56,529::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384436996, 'last-update-local-ts': 1384436996.5280211, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:56,529::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:49:56,530::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384436998, 'last-update-local-ts': 1384436996.5280211, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:49:56,530::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:50:06,614::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:50:06,877::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:50:06,941::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:50:07,003::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:50:07,003::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:50:07,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:50:07,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:07,010::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:50:07,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:50:07,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 15:50:07,011::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:07,012::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437007|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437007 (Thu Nov 14 15:50:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:50:07,012::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:50:07,022::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:50:07,022::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:50:07,022::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437007, 'last-update-local-ts': 1384437007.022064, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:07,022::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:50:07,023::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437009, 'last-update-local-ts': 1384437007.022064, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:07,023::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:50:17,123::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:50:17,381::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:50:17,464::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:50:17,524::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:50:17,524::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:50:17,525::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:50:17,525::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:17,525::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:50:17,525::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:50:17,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 15:50:17,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:17,526::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437017|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437017 (Thu Nov 14 15:50:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:50:17,527::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:50:17,535::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:50:17,535::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:50:17,535::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437017, 'last-update-local-ts': 1384437017.5350521, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:17,535::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:50:17,536::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437019, 'last-update-local-ts': 1384437017.5350521, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:17,536::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:50:27,615::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:50:27,871::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:50:27,951::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:50:28,012::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:50:28,012::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:50:28,013::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:50:28,013::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:28,013::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:50:28,013::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:50:28,014::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 15:50:28,014::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:28,014::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437028|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437028 (Thu Nov 14 15:50:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:50:28,015::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:50:28,024::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:50:28,024::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:50:28,025::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437028, 'last-update-local-ts': 1384437028.0245149, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:28,025::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:50:28,025::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437030, 'last-update-local-ts': 1384437028.0245149, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:28,025::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:50:38,110::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:50:38,359::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:50:38,438::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:50:38,497::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:50:38,498::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:50:38,498::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:50:38,498::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:38,499::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 15:50:38,499::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:50:38,499::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 15:50:38,499::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:38,501::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437038|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437038 (Thu Nov 14 15:50:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:50:38,501::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:50:38,510::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:50:38,510::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:50:38,510::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437038, 'last-update-local-ts': 1384437038.5100939, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:38,510::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:50:38,511::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437040, 'last-update-local-ts': 1384437038.5100939, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:38,511::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:50:48,586::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:50:48,837::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:50:48,918::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:50:48,985::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:50:48,985::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:50:48,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:50:48,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:48,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:50:48,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:50:48,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 15:50:48,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:48,987::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437048|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437048 (Thu Nov 14 15:50:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:50:48,987::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:50:48,995::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:50:48,995::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:50:48,996::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437048, 'last-update-local-ts': 1384437048.995419, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:48,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:50:48,996::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437051, 'last-update-local-ts': 1384437048.995419, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:48,996::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:50:59,070::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:50:59,336::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:50:59,399::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:50:59,459::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:50:59,459::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:50:59,459::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:50:59,460::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:59,460::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:50:59,460::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:50:59,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 15:50:59,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:50:59,462::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437059|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437059 (Thu Nov 14 15:50:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:50:59,462::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:50:59,470::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:50:59,471::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:50:59,471::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437059, 'last-update-local-ts': 1384437059.4708359, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:59,471::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:50:59,471::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437061, 'last-update-local-ts': 1384437059.4708359, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:50:59,472::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:51:09,571::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:51:09,822::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:51:09,912::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:51:09,974::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:51:09,974::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:51:09,975::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:51:09,975::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:09,975::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 15:51:09,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:51:09,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 15:51:09,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:09,977::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437069|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437069 (Thu Nov 14 15:51:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:51:09,977::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:51:09,985::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:51:09,986::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:51:09,986::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437069, 'last-update-local-ts': 1384437069.985801, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:09,986::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:51:09,986::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437072, 'last-update-local-ts': 1384437069.985801, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:09,987::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:51:20,068::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:51:20,317::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:51:20,402::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:51:20,463::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:51:20,463::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:51:20,470::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:51:20,470::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:20,470::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:51:20,471::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:51:20,471::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 15:51:20,471::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:20,472::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437080|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437080 (Thu Nov 14 15:51:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:51:20,472::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:51:20,482::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:51:20,482::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:51:20,482::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437080, 'last-update-local-ts': 1384437080.4819951, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:20,482::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:51:20,483::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437082, 'last-update-local-ts': 1384437080.4819951, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:20,483::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:51:30,561::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:51:30,814::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:51:30,898::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:51:30,959::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:51:30,959::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:51:30,960::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:51:30,960::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:30,960::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:51:30,961::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:51:30,961::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 15:51:30,961::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:30,962::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437090|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437090 (Thu Nov 14 15:51:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:51:30,962::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:51:30,971::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:51:30,971::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:51:30,971::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437090, 'last-update-local-ts': 1384437090.9709589, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:30,973::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:51:30,973::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437092, 'last-update-local-ts': 1384437090.9709589, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:30,973::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:51:41,043::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:51:41,290::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:51:41,370::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:51:41,439::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:51:41,439::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:51:41,440::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:51:41,440::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:41,440::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:51:41,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:51:41,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 15:51:41,441::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:41,442::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437101|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437101 (Thu Nov 14 15:51:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:51:41,442::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:51:41,459::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:51:41,459::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:51:41,460::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437101, 'last-update-local-ts': 1384437101.4593911, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:41,460::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:51:41,460::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437103, 'last-update-local-ts': 1384437101.4593911, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:41,460::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:51:51,548::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:51:51,951::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:51:52,113::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:51:52,196::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:51:52,196::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:51:52,197::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:51:52,197::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:52,197::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7033 >MainThread::INFO::2013-11-14 15:51:52,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:51:52,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 15:51:52,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:51:52,199::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437112|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437112 (Thu Nov 14 15:51:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7033 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:51:52,200::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:51:52,226::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:51:52,227::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:51:52,227::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437112, 'last-update-local-ts': 1384437112.2266469, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:52,227::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:51:52,227::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437113, 'last-update-local-ts': 1384437112.2266469, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:51:52,228::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:52:02,302::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:52:02,552::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:52:02,623::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:52:02,683::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:52:02,683::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:52:02,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:52:02,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:02,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:52:02,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:52:02,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 15:52:02,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:02,685::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437122|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437122 (Thu Nov 14 15:52:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:52:02,686::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:52:02,692::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:52:02,693::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:52:02,693::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437122, 'last-update-local-ts': 1384437122.6925499, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:02,693::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:52:02,693::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437124, 'last-update-local-ts': 1384437122.6925499, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:02,693::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:52:12,776::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:52:13,030::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:52:13,125::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:52:13,186::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:52:13,186::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:52:13,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:52:13,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:13,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:52:13,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:52:13,188::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 15:52:13,188::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:13,188::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437133|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437133 (Thu Nov 14 15:52:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:52:13,189::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:52:13,196::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:52:13,197::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:52:13,197::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437133, 'last-update-local-ts': 1384437133.196708, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:13,197::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:52:13,197::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437134, 'last-update-local-ts': 1384437133.196708, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:13,198::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:52:23,295::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:52:23,547::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:52:23,624::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:52:23,691::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:52:23,692::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:52:23,692::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:52:23,692::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:23,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:52:23,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:52:23,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 15:52:23,694::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:23,695::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437143|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437143 (Thu Nov 14 15:52:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:52:23,695::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:52:23,703::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:52:23,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:52:23,704::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437143, 'last-update-local-ts': 1384437143.7038419, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:23,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:52:23,704::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437145, 'last-update-local-ts': 1384437143.7038419, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:23,705::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:52:33,790::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:52:34,043::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:52:34,123::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:52:34,184::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:52:34,184::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:52:34,185::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:52:34,185::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:34,185::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:52:34,185::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:52:34,186::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 15:52:34,186::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:34,186::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437154|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437154 (Thu Nov 14 15:52:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:52:34,187::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:52:34,196::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:52:34,197::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:52:34,197::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437154, 'last-update-local-ts': 1384437154.1967261, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:34,197::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:52:34,197::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437155, 'last-update-local-ts': 1384437154.1967261, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:34,198::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:52:44,271::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:52:44,518::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:52:44,581::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:52:44,639::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:52:44,640::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:52:44,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:52:44,640::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:44,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 15:52:44,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:52:44,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 15:52:44,642::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:44,643::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437164|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437164 (Thu Nov 14 15:52:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:52:44,643::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:52:44,650::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:52:44,650::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:52:44,650::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437164, 'last-update-local-ts': 1384437164.650306, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:44,651::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:52:44,651::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437166, 'last-update-local-ts': 1384437164.650306, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:44,651::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:52:54,756::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:52:55,005::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:52:55,098::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:52:55,173::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:52:55,174::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:52:55,174::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:52:55,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:55,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:52:55,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:52:55,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 15:52:55,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:52:55,176::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437175|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437175 (Thu Nov 14 15:52:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:52:55,176::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:52:55,188::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:52:55,188::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:52:55,188::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437175, 'last-update-local-ts': 1384437175.1881731, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:55,189::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:52:55,189::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437176, 'last-update-local-ts': 1384437175.1881731, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:52:55,189::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:53:05,271::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:53:05,525::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:53:05,610::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:53:05,683::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:53:05,684::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:53:05,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:53:05,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:05,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:53:05,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:53:05,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 15:53:05,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:05,686::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437185|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437185 (Thu Nov 14 15:53:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:53:05,686::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:53:05,694::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:53:05,694::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:53:05,694::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437185, 'last-update-local-ts': 1384437185.694144, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:05,694::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:53:05,695::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437187, 'last-update-local-ts': 1384437185.694144, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:05,695::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:53:15,811::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:53:16,064::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:53:16,144::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:53:16,206::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:53:16,206::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:53:16,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:53:16,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:16,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:53:16,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:53:16,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 15:53:16,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:16,209::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437196|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437196 (Thu Nov 14 15:53:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:53:16,209::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:53:16,218::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:53:16,218::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:53:16,218::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437196, 'last-update-local-ts': 1384437196.2180459, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:16,218::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:53:16,219::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437197, 'last-update-local-ts': 1384437196.2180459, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:16,219::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:53:26,292::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:53:26,635::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:53:26,738::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:53:26,801::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:53:26,802::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:53:26,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:53:26,808::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:26,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:53:26,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:53:26,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 15:53:26,809::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:26,810::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437206|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437206 (Thu Nov 14 15:53:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:53:26,810::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:53:26,820::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:53:26,820::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:53:26,820::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437206, 'last-update-local-ts': 1384437206.8202181, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:26,821::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:53:26,821::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437207, 'last-update-local-ts': 1384437206.8202181, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:26,821::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:53:36,898::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:53:37,163::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:53:37,213::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:53:37,272::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:53:37,272::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:53:37,273::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:53:37,273::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:37,273::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:53:37,274::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:53:37,274::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 15:53:37,274::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:37,275::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437217|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437217 (Thu Nov 14 15:53:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:53:37,275::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:53:37,284::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:53:37,284::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:53:37,285::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437217, 'last-update-local-ts': 1384437217.2845089, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:37,285::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:53:37,285::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437218, 'last-update-local-ts': 1384437217.2845089, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:37,285::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:53:47,379::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:53:47,629::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:53:47,716::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:53:47,777::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:53:47,777::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:53:47,778::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:53:47,778::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:47,778::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:53:47,778::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:53:47,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.27 >MainThread::INFO::2013-11-14 15:53:47,779::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:47,779::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437227|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437227 (Thu Nov 14 15:53:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.27 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:53:47,780::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:53:47,788::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:53:47,789::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:53:47,789::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437227, 'last-update-local-ts': 1384437227.7887001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:47,790::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:53:47,790::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437228, 'last-update-local-ts': 1384437227.7887001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:47,790::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:53:57,866::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:53:58,115::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:53:58,220::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:53:58,281::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:53:58,281::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:53:58,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:53:58,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:58,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:53:58,289::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:53:58,289::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 15:53:58,289::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:53:58,290::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437238|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437238 (Thu Nov 14 15:53:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:53:58,290::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:53:58,300::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:53:58,300::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:53:58,300::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437238, 'last-update-local-ts': 1384437238.299896, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:58,300::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:53:58,300::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437239, 'last-update-local-ts': 1384437238.299896, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:53:58,301::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:54:08,386::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:54:08,640::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:54:08,724::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:54:08,785::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:54:08,785::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:54:08,786::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:54:08,786::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:08,786::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:54:08,786::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:54:08,787::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.195 >MainThread::INFO::2013-11-14 15:54:08,787::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:08,788::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437248|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437248 (Thu Nov 14 15:54:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.195 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:54:08,788::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:54:08,800::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:54:08,800::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:54:08,801::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437248, 'last-update-local-ts': 1384437248.8004179, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:08,801::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:54:08,801::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437249, 'last-update-local-ts': 1384437248.8004179, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:08,802::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:54:18,877::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:54:19,126::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:54:19,187::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:54:19,248::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:54:19,249::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:54:19,249::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:54:19,249::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:19,250::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:54:19,250::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:54:19,250::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 15:54:19,251::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:19,251::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437259|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437259 (Thu Nov 14 15:54:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:54:19,251::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:54:19,260::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:54:19,260::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:54:19,261::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437259, 'last-update-local-ts': 1384437259.2603951, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:19,261::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:54:19,261::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437260, 'last-update-local-ts': 1384437259.2603951, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:19,261::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:54:29,350::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:54:29,605::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:54:29,694::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:54:29,754::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:54:29,754::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:54:29,760::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:54:29,761::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:29,761::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:54:29,761::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:54:29,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 15:54:29,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:29,762::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437269|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437269 (Thu Nov 14 15:54:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:54:29,762::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:54:29,771::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:54:29,772::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:54:29,772::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437269, 'last-update-local-ts': 1384437269.7717061, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:29,772::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:54:29,772::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437270, 'last-update-local-ts': 1384437269.7717061, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:29,773::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:54:39,851::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:54:40,103::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:54:40,187::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:54:40,247::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:54:40,248::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:54:40,248::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:54:40,248::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:40,249::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 15:54:40,249::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:54:40,249::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 15:54:40,250::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:40,251::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437280|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437280 (Thu Nov 14 15:54:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:54:40,251::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:54:40,260::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:54:40,260::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:54:40,260::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437280, 'last-update-local-ts': 1384437280.2601449, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:40,261::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:54:40,261::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437281, 'last-update-local-ts': 1384437280.2601449, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:40,261::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:54:50,336::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:54:50,584::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:54:50,672::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:54:50,744::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:54:50,744::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:54:50,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:54:50,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:50,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:54:50,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:54:50,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 15:54:50,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:54:50,747::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437290|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437290 (Thu Nov 14 15:54:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:54:50,747::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:54:50,755::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:54:50,756::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:54:50,756::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437290, 'last-update-local-ts': 1384437290.7557471, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:50,756::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:54:50,756::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437291, 'last-update-local-ts': 1384437290.7557471, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:54:50,757::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:55:00,830::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:55:01,084::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:55:01,142::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:55:01,201::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:55:01,201::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:55:01,202::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:55:01,202::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:01,202::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:55:01,202::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:55:01,203::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.28 >MainThread::INFO::2013-11-14 15:55:01,203::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:01,204::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437301|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437301 (Thu Nov 14 15:55:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.28 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:55:01,204::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:55:01,212::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:55:01,212::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:55:01,213::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437301, 'last-update-local-ts': 1384437301.212466, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:01,213::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:55:01,213::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437301, 'last-update-local-ts': 1384437301.212466, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:01,213::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:55:11,322::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:55:11,577::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:55:11,712::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:55:11,772::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:55:11,773::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:55:11,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:55:11,773::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:11,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:55:11,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:55:11,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 15:55:11,774::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:11,775::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437311|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437311 (Thu Nov 14 15:55:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:55:11,776::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:55:11,784::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:55:11,785::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:55:11,785::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437311, 'last-update-local-ts': 1384437311.784842, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:11,785::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:55:11,785::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437312, 'last-update-local-ts': 1384437311.784842, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:11,786::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:55:21,867::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:55:22,132::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:55:22,279::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:55:22,340::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:55:22,340::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:55:22,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:55:22,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:22,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7037 >MainThread::INFO::2013-11-14 15:55:22,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:55:22,342::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 15:55:22,342::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:22,343::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437322|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437322 (Thu Nov 14 15:55:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7037 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:55:22,343::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:55:22,350::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:55:22,350::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:55:22,350::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437322, 'last-update-local-ts': 1384437322.350122, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:22,350::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:55:22,351::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437322, 'last-update-local-ts': 1384437322.350122, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:22,351::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:55:32,427::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:55:32,677::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:55:32,766::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:55:32,839::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:55:32,839::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:55:32,839::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:55:32,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:32,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 15:55:32,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:55:32,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 15:55:32,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:32,841::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437332|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437332 (Thu Nov 14 15:55:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:55:32,841::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:55:32,848::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:55:32,848::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:55:32,849::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437332, 'last-update-local-ts': 1384437332.848479, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:32,849::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:55:32,849::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437333, 'last-update-local-ts': 1384437332.848479, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:32,849::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:55:42,919::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:55:43,173::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:55:43,241::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:55:43,301::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:55:43,301::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:55:43,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:55:43,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:43,302::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:55:43,303::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:55:43,303::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 15:55:43,303::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:43,304::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437343|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437343 (Thu Nov 14 15:55:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:55:43,304::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:55:43,312::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:55:43,313::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:55:43,313::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437343, 'last-update-local-ts': 1384437343.3125961, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:43,313::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:55:43,313::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437343, 'last-update-local-ts': 1384437343.3125961, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:43,313::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:55:53,401::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:55:53,650::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:55:53,741::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:55:53,802::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:55:53,802::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:55:53,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:55:53,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:53,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:55:53,804::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:55:53,804::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 15:55:53,804::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:55:53,805::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437353|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437353 (Thu Nov 14 15:55:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:55:53,805::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:55:53,814::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:55:53,814::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:55:53,814::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437353, 'last-update-local-ts': 1384437353.8141019, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:53,814::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:55:53,815::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437354, 'last-update-local-ts': 1384437353.8141019, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:55:53,815::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:56:03,891::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:56:04,154::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:56:04,245::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:56:04,318::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:56:04,318::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:56:04,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:56:04,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:04,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:56:04,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:56:04,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 15:56:04,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:04,320::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437364|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437364 (Thu Nov 14 15:56:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:56:04,320::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:56:04,328::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:56:04,328::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:56:04,329::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437364, 'last-update-local-ts': 1384437364.3284471, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:04,329::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:56:04,329::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437364, 'last-update-local-ts': 1384437364.3284471, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:04,329::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:56:14,416::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:56:14,669::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:56:14,755::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:56:14,816::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:56:14,817::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:56:14,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:56:14,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:14,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:56:14,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:56:14,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 15:56:14,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:14,820::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437374|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437374 (Thu Nov 14 15:56:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:56:14,820::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:56:14,829::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:56:14,829::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:56:14,829::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437374, 'last-update-local-ts': 1384437374.8291919, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:14,830::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:56:14,830::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437375, 'last-update-local-ts': 1384437374.8291919, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:14,830::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:56:24,906::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:56:25,160::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:56:25,220::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:56:25,287::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:56:25,287::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:56:25,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:56:25,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:25,288::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:56:25,289::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:56:25,289::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 15:56:25,289::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:25,289::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437385|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437385 (Thu Nov 14 15:56:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:56:25,290::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:56:25,298::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:56:25,298::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:56:25,298::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437385, 'last-update-local-ts': 1384437385.2981031, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:25,298::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:56:25,299::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437385, 'last-update-local-ts': 1384437385.2981031, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:25,299::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:56:35,386::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:56:35,638::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:56:35,738::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:56:35,798::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:56:35,798::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:56:35,798::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:56:35,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:35,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:56:35,799::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:56:35,800::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 15:56:35,800::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:35,801::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437395|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437395 (Thu Nov 14 15:56:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:56:35,802::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:56:35,811::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:56:35,811::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:56:35,811::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437395, 'last-update-local-ts': 1384437395.811146, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:35,812::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:56:35,812::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437396, 'last-update-local-ts': 1384437395.811146, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:35,812::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:56:45,903::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:56:46,154::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:56:46,237::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:56:46,303::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:56:46,304::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:56:46,304::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:56:46,304::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:46,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:56:46,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:56:46,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 15:56:46,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:46,306::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437406|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437406 (Thu Nov 14 15:56:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:56:46,306::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:56:46,315::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:56:46,315::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:56:46,315::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437406, 'last-update-local-ts': 1384437406.3150251, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:46,315::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:56:46,316::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437406, 'last-update-local-ts': 1384437406.3150251, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:46,316::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:56:56,402::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:56:56,661::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:56:56,749::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:56:56,809::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:56:56,810::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:56:56,810::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:56:56,811::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:56,811::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:56:56,811::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:56:56,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 15:56:56,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:56:56,812::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437416|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437416 (Thu Nov 14 15:56:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:56:56,812::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:56:56,822::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:56:56,823::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:56:56,823::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437416, 'last-update-local-ts': 1384437416.822552, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:56,823::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:56:56,823::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437416, 'last-update-local-ts': 1384437416.822552, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:56:56,823::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:57:06,922::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:57:07,180::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:57:07,258::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:57:07,318::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:57:07,318::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:57:07,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:57:07,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:07,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:57:07,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:57:07,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 15:57:07,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:07,322::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437427|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437427 (Thu Nov 14 15:57:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:57:07,322::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:57:07,331::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:57:07,331::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:57:07,332::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437427, 'last-update-local-ts': 1384437427.3313229, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:07,332::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:57:07,332::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437427, 'last-update-local-ts': 1384437427.3313229, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:07,332::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:57:17,405::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:57:17,654::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:57:17,714::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:57:17,774::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:57:17,774::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:57:17,775::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:57:17,775::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:17,775::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:57:17,776::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:57:17,776::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 15:57:17,776::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:17,776::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437437|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437437 (Thu Nov 14 15:57:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:57:17,777::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:57:17,785::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:57:17,785::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:57:17,785::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437437, 'last-update-local-ts': 1384437437.7851479, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:17,785::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:57:17,786::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437437, 'last-update-local-ts': 1384437437.7851479, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:17,786::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:57:27,871::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:57:28,135::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:57:28,247::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:57:28,310::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:57:28,310::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:57:28,311::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:57:28,311::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:28,311::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:57:28,312::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:57:28,312::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 15:57:28,312::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:28,313::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437448|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437448 (Thu Nov 14 15:57:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:57:28,314::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:57:28,322::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:57:28,323::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:57:28,323::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437448, 'last-update-local-ts': 1384437448.322638, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:28,323::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:57:28,323::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437448, 'last-update-local-ts': 1384437448.322638, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:28,323::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:57:38,425::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:57:38,689::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:57:38,767::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:57:38,829::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:57:38,829::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:57:38,830::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:57:38,830::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:38,830::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:57:38,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:57:38,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 15:57:38,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:38,831::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437458|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437458 (Thu Nov 14 15:57:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:57:38,832::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:57:38,840::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:57:38,841::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:57:38,841::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437458, 'last-update-local-ts': 1384437458.840661, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:38,841::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:57:38,841::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437458, 'last-update-local-ts': 1384437458.840661, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:38,841::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:57:48,914::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:57:49,171::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:57:49,261::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:57:49,323::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:57:49,323::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:57:49,330::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:57:49,330::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:49,330::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:57:49,331::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:57:49,331::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 15:57:49,331::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:49,332::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437469|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437469 (Thu Nov 14 15:57:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:57:49,332::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:57:49,341::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:57:49,342::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:57:49,342::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437469, 'last-update-local-ts': 1384437469.341836, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:49,342::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:57:49,342::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437469, 'last-update-local-ts': 1384437469.341836, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:49,343::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:57:59,427::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:57:59,692::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:57:59,783::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:57:59,844::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:57:59,844::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:57:59,845::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:57:59,845::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:59,845::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:57:59,846::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:57:59,846::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 15:57:59,846::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:57:59,847::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437479|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437479 (Thu Nov 14 15:57:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:57:59,847::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:57:59,857::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:57:59,857::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:57:59,857::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437479, 'last-update-local-ts': 1384437479.8572869, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:59,858::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:57:59,858::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437479, 'last-update-local-ts': 1384437479.8572869, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:57:59,858::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:58:09,935::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:58:10,200::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:58:10,287::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:58:10,355::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:58:10,355::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:58:10,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:58:10,356::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:10,356::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:58:10,356::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:58:10,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 15:58:10,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:10,357::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437490|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437490 (Thu Nov 14 15:58:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:58:10,357::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:58:10,366::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:58:10,366::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:58:10,366::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437490, 'last-update-local-ts': 1384437490.3663311, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:10,367::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:58:10,367::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437490, 'last-update-local-ts': 1384437490.3663311, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:10,367::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:58:20,440::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:58:20,696::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:58:20,750::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:58:20,810::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:58:20,810::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:58:20,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:58:20,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:20,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:58:20,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:58:20,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 15:58:20,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:20,819::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437500|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437500 (Thu Nov 14 15:58:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:58:20,819::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:58:20,829::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:58:20,829::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:58:20,830::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437500, 'last-update-local-ts': 1384437500.8293209, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:20,830::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:58:20,830::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437500, 'last-update-local-ts': 1384437500.8293209, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:20,830::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:58:30,910::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:58:31,159::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:58:31,262::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:58:31,323::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:58:31,323::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:58:31,324::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:58:31,324::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:31,324::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:58:31,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:58:31,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 15:58:31,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:31,326::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437511|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437511 (Thu Nov 14 15:58:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:58:31,327::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:58:31,333::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:58:31,334::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:58:31,334::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437511, 'last-update-local-ts': 1384437511.333833, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:31,334::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:58:31,334::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437510, 'last-update-local-ts': 1384437511.333833, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:31,335::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:58:41,416::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:58:41,665::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:58:41,753::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:58:41,813::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:58:41,814::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:58:41,820::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:58:41,820::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:41,821::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:58:41,821::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:58:41,821::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 15:58:41,822::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:41,822::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437521|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437521 (Thu Nov 14 15:58:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:58:41,822::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:58:41,831::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:58:41,831::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:58:41,831::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437521, 'last-update-local-ts': 1384437521.831167, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:41,832::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:58:41,832::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437521, 'last-update-local-ts': 1384437521.831167, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:41,832::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:58:51,913::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:58:52,173::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:58:52,255::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:58:52,316::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:58:52,316::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:58:52,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:58:52,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:52,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7034 >MainThread::INFO::2013-11-14 15:58:52,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:58:52,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 15:58:52,318::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:58:52,319::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437532|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437532 (Thu Nov 14 15:58:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7034 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:58:52,320::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:58:52,330::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:58:52,331::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:58:52,332::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437532, 'last-update-local-ts': 1384437532.3306179, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:52,332::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:58:52,332::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437531, 'last-update-local-ts': 1384437532.3306179, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:58:52,333::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:59:02,412::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:59:02,680::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:59:02,743::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:59:02,804::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:59:02,804::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:59:02,805::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:59:02,805::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:02,805::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:59:02,806::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:59:02,806::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 15:59:02,806::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:02,807::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437542|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437542 (Thu Nov 14 15:59:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:59:02,807::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:59:02,813::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:59:02,814::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:59:02,814::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437542, 'last-update-local-ts': 1384437542.813679, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:02,814::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:59:02,814::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437542, 'last-update-local-ts': 1384437542.813679, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:02,815::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:59:12,902::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:59:13,150::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:59:13,238::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:59:13,298::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:59:13,298::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:59:13,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:59:13,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:13,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:59:13,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:59:13,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.49 >MainThread::INFO::2013-11-14 15:59:13,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:13,307::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437553|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437553 (Thu Nov 14 15:59:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.49 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:59:13,307::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:59:13,316::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:59:13,316::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:59:13,317::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437553, 'last-update-local-ts': 1384437553.3164971, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:13,317::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:59:13,317::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437552, 'last-update-local-ts': 1384437553.3164971, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:13,317::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:59:23,412::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:59:23,665::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:59:23,755::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:59:23,814::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:59:23,815::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:59:23,815::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:59:23,815::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:23,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:59:23,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:59:23,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.415 >MainThread::INFO::2013-11-14 15:59:23,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:23,818::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437563|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437563 (Thu Nov 14 15:59:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.415 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:59:23,818::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:59:23,824::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:59:23,825::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:59:23,825::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437563, 'last-update-local-ts': 1384437563.8248091, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:23,825::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:59:23,825::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437563, 'last-update-local-ts': 1384437563.8248091, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:23,826::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:59:33,901::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:59:34,155::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:59:34,241::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:59:34,315::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:59:34,315::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:59:34,315::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:59:34,316::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:34,316::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:59:34,316::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:59:34,316::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.35 >MainThread::INFO::2013-11-14 15:59:34,317::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:34,317::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437574|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437574 (Thu Nov 14 15:59:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.35 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:59:34,317::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:59:34,326::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:59:34,326::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:59:34,326::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437574, 'last-update-local-ts': 1384437574.326021, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:34,326::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:59:34,327::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437573, 'last-update-local-ts': 1384437574.326021, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:34,327::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:59:44,392::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:59:44,643::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:59:44,706::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:59:44,767::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:59:44,767::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:59:44,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:59:44,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:44,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 15:59:44,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:59:44,769::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 15:59:44,769::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:44,770::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437584|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437584 (Thu Nov 14 15:59:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:59:44,770::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:59:44,786::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:59:44,786::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:59:44,786::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437584, 'last-update-local-ts': 1384437584.7859499, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:44,786::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:59:44,787::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437584, 'last-update-local-ts': 1384437584.7859499, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:44,787::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 15:59:54,897::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 15:59:55,201::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 15:59:55,316::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 15:59:55,389::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 15:59:55,389::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 15:59:55,390::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 15:59:55,390::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:55,391::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 15:59:55,391::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 15:59:55,391::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.395 >MainThread::INFO::2013-11-14 15:59:55,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 15:59:55,392::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437595|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437595 (Thu Nov 14 15:59:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.395 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 15:59:55,392::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 15:59:55,401::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 15:59:55,401::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 15:59:55,402::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437595, 'last-update-local-ts': 1384437595.4013929, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:55,402::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 15:59:55,402::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437594, 'last-update-local-ts': 1384437595.4013929, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 15:59:55,402::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:00:05,491::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:00:05,750::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:00:05,864::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:00:05,925::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:00:05,925::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:00:05,925::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:00:05,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:05,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:00:05,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:00:05,926::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 16:00:05,927::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:05,927::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437605|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437605 (Thu Nov 14 16:00:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:00:05,927::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:00:05,936::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:00:05,936::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:00:05,936::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437605, 'last-update-local-ts': 1384437605.936296, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:05,937::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:00:05,937::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437605, 'last-update-local-ts': 1384437605.936296, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:05,937::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:00:16,042::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:00:16,294::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:00:16,394::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:00:16,461::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:00:16,461::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:00:16,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:00:16,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:16,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:00:16,463::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:00:16,463::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 16:00:16,463::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:16,464::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437616|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437616 (Thu Nov 14 16:00:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:00:16,464::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:00:16,471::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:00:16,472::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:00:16,472::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437616, 'last-update-local-ts': 1384437616.4715979, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:16,472::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:00:16,472::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437615, 'last-update-local-ts': 1384437616.4715979, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:16,472::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:00:26,558::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:00:26,811::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:00:26,898::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:00:26,958::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:00:26,958::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:00:26,965::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:00:26,965::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:26,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:00:26,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:00:26,966::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 16:00:26,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:26,967::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437626|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437626 (Thu Nov 14 16:00:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:00:26,967::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:00:26,977::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:00:26,977::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:00:26,977::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437626, 'last-update-local-ts': 1384437626.9768929, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:26,977::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:00:26,977::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437625, 'last-update-local-ts': 1384437626.9768929, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:26,978::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:00:37,081::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:00:37,340::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:00:37,407::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:00:37,467::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:00:37,467::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:00:37,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:00:37,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:37,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:00:37,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:00:37,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 16:00:37,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:37,470::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437637|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437637 (Thu Nov 14 16:00:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:00:37,471::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:00:37,479::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:00:37,479::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:00:37,479::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437637, 'last-update-local-ts': 1384437637.478919, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:37,479::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:00:37,479::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437636, 'last-update-local-ts': 1384437637.478919, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:37,480::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:00:47,561::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:00:47,828::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:00:47,890::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:00:47,956::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:00:47,956::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:00:47,957::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:00:47,957::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:47,957::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:00:47,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:00:47,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 16:00:47,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:47,958::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437647|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437647 (Thu Nov 14 16:00:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:00:47,959::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:00:47,967::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:00:47,967::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:00:47,967::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437647, 'last-update-local-ts': 1384437647.9669721, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:47,967::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:00:47,968::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437646, 'last-update-local-ts': 1384437647.9669721, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:47,968::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:00:58,034::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:00:58,290::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:00:58,382::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:00:58,443::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:00:58,444::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:00:58,450::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:00:58,450::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:58,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:00:58,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:00:58,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 16:00:58,451::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:00:58,452::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437658|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437658 (Thu Nov 14 16:00:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:00:58,452::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:00:58,462::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:00:58,462::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:00:58,462::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437658, 'last-update-local-ts': 1384437658.4620039, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:58,462::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:00:58,463::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437657, 'last-update-local-ts': 1384437658.4620039, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:00:58,463::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:01:08,572::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:01:08,841::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:01:08,980::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:01:09,040::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:01:09,041::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:01:09,041::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:01:09,041::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:09,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6988 >MainThread::INFO::2013-11-14 16:01:09,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:01:09,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 16:01:09,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:09,044::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437669|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437669 (Thu Nov 14 16:01:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=6988 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:01:09,044::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:01:09,052::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:01:09,053::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:01:09,053::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437669, 'last-update-local-ts': 1384437669.052633, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:09,053::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:01:09,053::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437667, 'last-update-local-ts': 1384437669.052633, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:09,053::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:01:19,140::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:01:19,394::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:01:19,493::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:01:19,554::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:01:19,554::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:01:19,555::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:01:19,555::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:19,555::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:01:19,555::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:01:19,556::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.315 >MainThread::INFO::2013-11-14 16:01:19,556::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:19,556::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437679|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437679 (Thu Nov 14 16:01:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.315 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:01:19,557::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:01:19,566::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:01:19,566::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:01:19,566::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437679, 'last-update-local-ts': 1384437679.5659781, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:19,566::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:01:19,567::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437678, 'last-update-local-ts': 1384437679.5659781, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:19,567::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:01:29,641::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:01:29,906::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:01:29,987::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:01:30,047::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:01:30,048::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:01:30,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:01:30,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:30,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:01:30,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:01:30,055::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.27 >MainThread::INFO::2013-11-14 16:01:30,056::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:30,056::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437690|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437690 (Thu Nov 14 16:01:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.27 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:01:30,056::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:01:30,066::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:01:30,066::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:01:30,066::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437690, 'last-update-local-ts': 1384437690.0660779, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:30,066::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:01:30,067::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437688, 'last-update-local-ts': 1384437690.0660779, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:30,067::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:01:40,141::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:01:40,394::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:01:40,454::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:01:40,515::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:01:40,515::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:01:40,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:01:40,516::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:40,517::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:01:40,517::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:01:40,517::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 16:01:40,517::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:40,519::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437700|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437700 (Thu Nov 14 16:01:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:01:40,519::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:01:40,528::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:01:40,528::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:01:40,528::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437700, 'last-update-local-ts': 1384437700.528132, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:40,528::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:01:40,529::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437699, 'last-update-local-ts': 1384437700.528132, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:40,529::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:01:50,626::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:01:50,879::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:01:50,962::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:01:51,032::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:01:51,032::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:01:51,033::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:01:51,033::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:51,033::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:01:51,034::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:01:51,034::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 16:01:51,034::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:01:51,035::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437711|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437711 (Thu Nov 14 16:01:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:01:51,035::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:01:51,043::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:01:51,044::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:01:51,044::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437711, 'last-update-local-ts': 1384437711.0436909, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:51,044::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:01:51,044::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437709, 'last-update-local-ts': 1384437711.0436909, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:01:51,045::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:02:01,131::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:02:01,385::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:02:01,467::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:02:01,552::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:02:01,552::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:02:01,558::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:02:01,558::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:01,559::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:02:01,559::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:02:01,559::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 16:02:01,560::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:01,560::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437721|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437721 (Thu Nov 14 16:02:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:02:01,560::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:02:01,570::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:02:01,570::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:02:01,571::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437721, 'last-update-local-ts': 1384437721.5704861, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:01,571::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:02:01,571::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437720, 'last-update-local-ts': 1384437721.5704861, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:01,571::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:02:11,687::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:02:11,939::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:02:12,014::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:02:12,075::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:02:12,075::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:02:12,076::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:02:12,076::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:12,076::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:02:12,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:02:12,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 16:02:12,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:12,078::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437732|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437732 (Thu Nov 14 16:02:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:02:12,078::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:02:12,088::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:02:12,089::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:02:12,089::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437732, 'last-update-local-ts': 1384437732.088774, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:12,089::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:02:12,089::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437730, 'last-update-local-ts': 1384437732.088774, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:12,090::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:02:22,162::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:02:22,423::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:02:22,503::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:02:22,570::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:02:22,571::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:02:22,571::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:02:22,571::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:22,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7034 >MainThread::INFO::2013-11-14 16:02:22,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:02:22,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.15 >MainThread::INFO::2013-11-14 16:02:22,572::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:22,573::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437742|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437742 (Thu Nov 14 16:02:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.15 >engine-health=vm-down >gateway=True >mem-free=7034 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:02:22,573::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:02:22,580::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:02:22,580::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:02:22,580::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437742, 'last-update-local-ts': 1384437742.5799119, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:22,580::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:02:22,580::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437741, 'last-update-local-ts': 1384437742.5799119, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:22,581::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:02:32,670::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:02:32,925::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:02:33,015::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:02:33,076::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:02:33,076::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:02:33,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:02:33,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:33,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:02:33,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:02:33,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.535 >MainThread::INFO::2013-11-14 16:02:33,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:33,079::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437753|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437753 (Thu Nov 14 16:02:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.535 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:02:33,079::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:02:33,085::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:02:33,086::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:02:33,086::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437753, 'last-update-local-ts': 1384437753.085789, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:33,086::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:02:33,086::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437751, 'last-update-local-ts': 1384437753.085789, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:33,087::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:02:43,180::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:02:43,429::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:02:43,513::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:02:43,574::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:02:43,574::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:02:43,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:02:43,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:43,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:02:43,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:02:43,576::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.455 >MainThread::INFO::2013-11-14 16:02:43,576::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:43,577::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437763|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437763 (Thu Nov 14 16:02:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.455 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:02:43,577::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:02:43,586::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:02:43,586::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:02:43,586::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437763, 'last-update-local-ts': 1384437763.586087, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:43,586::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:02:43,587::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437762, 'last-update-local-ts': 1384437763.586087, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:43,587::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:02:53,662::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:02:53,915::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:02:54,003::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:02:54,070::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:02:54,070::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:02:54,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:02:54,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:54,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:02:54,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:02:54,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.385 >MainThread::INFO::2013-11-14 16:02:54,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:02:54,079::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437774|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437774 (Thu Nov 14 16:02:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.385 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:02:54,079::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:02:54,088::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:02:54,088::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:02:54,088::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437774, 'last-update-local-ts': 1384437774.0879021, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:54,088::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:02:54,088::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437772, 'last-update-local-ts': 1384437774.0879021, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:02:54,089::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:03:04,162::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:03:04,415::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:03:04,500::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:03:04,561::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:03:04,562::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:03:04,562::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:03:04,562::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:04,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:03:04,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:03:04,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.325 >MainThread::INFO::2013-11-14 16:03:04,564::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:04,565::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437784|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437784 (Thu Nov 14 16:03:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.325 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:03:04,565::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:03:04,573::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:03:04,574::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:03:04,574::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437784, 'last-update-local-ts': 1384437784.5735631, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:04,574::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:03:04,574::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437782, 'last-update-local-ts': 1384437784.5735631, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:04,574::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:03:14,649::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:03:14,905::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:03:14,963::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:03:15,023::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:03:15,024::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:03:15,024::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:03:15,024::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:15,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:03:15,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:03:15,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.275 >MainThread::INFO::2013-11-14 16:03:15,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:15,026::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437795|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437795 (Thu Nov 14 16:03:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.275 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:03:15,026::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:03:15,034::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:03:15,035::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:03:15,035::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437795, 'last-update-local-ts': 1384437795.0347199, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:15,035::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:03:15,035::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437793, 'last-update-local-ts': 1384437795.0347199, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:15,036::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:03:25,124::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:03:25,370::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:03:25,458::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:03:25,517::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:03:25,518::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:03:25,524::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:03:25,525::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:25,525::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:03:25,525::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:03:25,525::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 16:03:25,526::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:25,526::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437805|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437805 (Thu Nov 14 16:03:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:03:25,526::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:03:25,537::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:03:25,537::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:03:25,537::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437805, 'last-update-local-ts': 1384437805.536927, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:25,537::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:03:25,537::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437803, 'last-update-local-ts': 1384437805.536927, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:25,538::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:03:35,634::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:03:35,886::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:03:35,982::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:03:36,041::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:03:36,041::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:03:36,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:03:36,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:36,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:03:36,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:03:36,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.195 >MainThread::INFO::2013-11-14 16:03:36,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:36,045::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437816|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437816 (Thu Nov 14 16:03:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.195 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:03:36,045::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:03:36,054::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:03:36,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:03:36,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437816, 'last-update-local-ts': 1384437816.054044, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:36,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:03:36,055::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437814, 'last-update-local-ts': 1384437816.054044, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:36,055::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:03:46,131::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:03:46,386::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:03:46,472::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:03:46,538::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:03:46,538::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:03:46,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:03:46,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:46,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:03:46,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:03:46,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.165 >MainThread::INFO::2013-11-14 16:03:46,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:46,541::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437826|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437826 (Thu Nov 14 16:03:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.165 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:03:46,541::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:03:46,549::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:03:46,550::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:03:46,550::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437826, 'last-update-local-ts': 1384437826.549845, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:46,550::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:03:46,550::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437824, 'last-update-local-ts': 1384437826.549845, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:46,551::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:03:56,625::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:03:56,879::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:03:56,964::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:03:57,031::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:03:57,031::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:03:57,032::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:03:57,032::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:57,032::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:03:57,033::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:03:57,033::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 16:03:57,033::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:03:57,034::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437837|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437837 (Thu Nov 14 16:03:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:03:57,034::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:03:57,043::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:03:57,044::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:03:57,044::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437837, 'last-update-local-ts': 1384437837.0437751, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:57,044::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:03:57,044::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437835, 'last-update-local-ts': 1384437837.0437751, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:03:57,045::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:04:07,126::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:04:07,396::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:04:07,458::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:04:07,533::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:04:07,533::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:04:07,534::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:04:07,534::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:07,534::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:04:07,535::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:04:07,535::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 16:04:07,535::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:07,535::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437847|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437847 (Thu Nov 14 16:04:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:04:07,536::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:04:07,544::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:04:07,545::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:04:07,545::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437847, 'last-update-local-ts': 1384437847.544646, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:07,545::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:04:07,545::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437845, 'last-update-local-ts': 1384437847.544646, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:07,545::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:04:17,619::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:04:17,871::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:04:17,976::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:04:18,046::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:04:18,046::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:04:18,052::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:04:18,053::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:18,053::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:04:18,053::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:04:18,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 16:04:18,054::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:18,054::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437858|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437858 (Thu Nov 14 16:04:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:04:18,055::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:04:18,064::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:04:18,064::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:04:18,065::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437858, 'last-update-local-ts': 1384437858.0643489, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:18,065::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:04:18,065::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437856, 'last-update-local-ts': 1384437858.0643489, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:18,065::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:04:28,151::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:04:28,404::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:04:28,488::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:04:28,550::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:04:28,550::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:04:28,551::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:04:28,551::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:28,551::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:04:28,552::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:04:28,552::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 16:04:28,552::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:28,553::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437868|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437868 (Thu Nov 14 16:04:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:04:28,553::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:04:28,562::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:04:28,562::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:04:28,562::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437868, 'last-update-local-ts': 1384437868.562022, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:28,562::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:04:28,563::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437866, 'last-update-local-ts': 1384437868.562022, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:28,563::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:04:38,637::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:04:38,895::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:04:38,983::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:04:39,051::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:04:39,051::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:04:39,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:04:39,052::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:39,052::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:04:39,052::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:04:39,053::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 16:04:39,053::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:39,053::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437879|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437879 (Thu Nov 14 16:04:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:04:39,053::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:04:39,062::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:04:39,063::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:04:39,063::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437879, 'last-update-local-ts': 1384437879.062659, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:39,063::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:04:39,063::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437876, 'last-update-local-ts': 1384437879.062659, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:39,063::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:04:49,179::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:04:49,440::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:04:49,525::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:04:49,585::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:04:49,585::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:04:49,585::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:04:49,586::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:49,586::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:04:49,586::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:04:49,587::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 16:04:49,587::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:04:49,588::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437889|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437889 (Thu Nov 14 16:04:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:04:49,588::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:04:49,596::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:04:49,596::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:04:49,596::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437889, 'last-update-local-ts': 1384437889.595943, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:49,596::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:04:49,596::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437887, 'last-update-local-ts': 1384437889.595943, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:04:49,597::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:04:59,675::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:04:59,937::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:04:59,998::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:05:00,064::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:05:00,064::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:05:00,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:05:00,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:00,066::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:05:00,066::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:05:00,066::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:05:00,066::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:00,067::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437900|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437900 (Thu Nov 14 16:05:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:05:00,067::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:05:00,075::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:05:00,076::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:05:00,076::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437900, 'last-update-local-ts': 1384437900.0757899, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:00,076::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:05:00,076::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437897, 'last-update-local-ts': 1384437900.0757899, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:00,077::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:05:10,164::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:05:10,414::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:05:10,506::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:05:10,566::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:05:10,566::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:05:10,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:05:10,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:10,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:05:10,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:05:10,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 16:05:10,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:10,575::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437910|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437910 (Thu Nov 14 16:05:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:05:10,575::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:05:10,585::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:05:10,585::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:05:10,585::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437910, 'last-update-local-ts': 1384437910.58516, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:10,586::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:05:10,586::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437908, 'last-update-local-ts': 1384437910.58516, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:10,586::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:05:20,681::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:05:20,934::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:05:21,022::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:05:21,081::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:05:21,081::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:05:21,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:05:21,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:21,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:05:21,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:05:21,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 16:05:21,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:21,085::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437921|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437921 (Thu Nov 14 16:05:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:05:21,085::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:05:21,093::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:05:21,093::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:05:21,093::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437921, 'last-update-local-ts': 1384437921.093276, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:21,094::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:05:21,094::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437918, 'last-update-local-ts': 1384437921.093276, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:21,094::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:05:31,187::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:05:31,481::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:05:31,563::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:05:31,637::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:05:31,637::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:05:31,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:05:31,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:31,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:05:31,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:05:31,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 16:05:31,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:31,639::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437931|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437931 (Thu Nov 14 16:05:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:05:31,639::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:05:31,648::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:05:31,648::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:05:31,648::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437931, 'last-update-local-ts': 1384437931.648077, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:31,648::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:05:31,649::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437929, 'last-update-local-ts': 1384437931.648077, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:31,649::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:05:41,727::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:05:41,997::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:05:42,049::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:05:42,108::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:05:42,108::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:05:42,109::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:05:42,109::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:42,109::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:05:42,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:05:42,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 16:05:42,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:42,110::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437942|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437942 (Thu Nov 14 16:05:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:05:42,111::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:05:42,121::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:05:42,122::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:05:42,122::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437942, 'last-update-local-ts': 1384437942.1215279, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:42,122::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:05:42,122::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437939, 'last-update-local-ts': 1384437942.1215279, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:42,122::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:05:52,200::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:05:52,459::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:05:52,550::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:05:52,610::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:05:52,611::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:05:52,611::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:05:52,611::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:52,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7034 >MainThread::INFO::2013-11-14 16:05:52,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:05:52,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 16:05:52,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:05:52,613::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437952|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437952 (Thu Nov 14 16:05:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7034 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:05:52,614::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:05:52,620::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:05:52,621::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:05:52,621::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437952, 'last-update-local-ts': 1384437952.620779, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:52,621::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:05:52,621::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437950, 'last-update-local-ts': 1384437952.620779, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:05:52,622::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:06:02,712::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:06:02,979::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:06:03,064::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:06:03,131::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:06:03,131::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:06:03,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:06:03,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:03,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:06:03,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:06:03,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 16:06:03,133::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:03,133::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437963|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437963 (Thu Nov 14 16:06:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:06:03,133::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:06:03,140::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:06:03,140::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:06:03,141::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437963, 'last-update-local-ts': 1384437963.1404071, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:03,141::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:06:03,141::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437960, 'last-update-local-ts': 1384437963.1404071, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:03,141::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:06:13,227::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:06:13,482::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:06:13,582::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:06:13,648::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:06:13,648::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:06:13,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:06:13,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:13,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:06:13,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:06:13,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:06:13,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:13,651::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437973|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437973 (Thu Nov 14 16:06:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:06:13,651::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:06:13,658::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:06:13,658::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:06:13,659::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437973, 'last-update-local-ts': 1384437973.6584821, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:13,659::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:06:13,659::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437971, 'last-update-local-ts': 1384437973.6584821, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:13,659::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:06:23,733::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:06:23,980::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:06:24,041::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:06:24,101::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:06:24,102::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:06:24,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:06:24,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:24,103::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:06:24,103::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:06:24,103::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 16:06:24,103::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:24,104::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437984|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437984 (Thu Nov 14 16:06:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:06:24,104::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:06:24,112::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:06:24,113::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:06:24,113::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437984, 'last-update-local-ts': 1384437984.112725, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:24,113::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:06:24,113::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437981, 'last-update-local-ts': 1384437984.112725, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:24,114::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:06:34,195::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:06:34,450::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:06:34,608::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:06:34,675::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:06:34,675::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:06:34,676::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:06:34,676::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:34,676::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:06:34,676::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:06:34,677::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 16:06:34,677::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:34,677::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384437994|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384437994 (Thu Nov 14 16:06:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:06:34,678::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:06:34,686::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:06:34,686::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:06:34,686::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384437994, 'last-update-local-ts': 1384437994.686228, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:34,687::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:06:34,687::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384437992, 'last-update-local-ts': 1384437994.686228, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:34,687::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:06:44,766::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:06:45,048::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:06:45,136::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:06:45,197::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:06:45,197::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:06:45,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:06:45,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:45,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:06:45,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:06:45,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 16:06:45,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:45,200::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438005|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438005 (Thu Nov 14 16:06:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:06:45,200::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:06:45,208::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:06:45,209::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:06:45,209::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438005, 'last-update-local-ts': 1384438005.208847, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:45,209::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:06:45,209::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438002, 'last-update-local-ts': 1384438005.208847, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:45,210::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:06:55,288::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:06:55,551::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:06:55,646::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:06:55,713::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:06:55,713::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:06:55,713::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:06:55,714::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:55,714::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:06:55,714::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:06:55,715::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 16:06:55,715::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:06:55,715::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438015|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438015 (Thu Nov 14 16:06:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:06:55,715::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:06:55,723::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:06:55,724::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:06:55,724::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438015, 'last-update-local-ts': 1384438015.7236171, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:55,724::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:06:55,724::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438012, 'last-update-local-ts': 1384438015.7236171, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:06:55,724::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:07:05,799::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:07:06,052::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:07:06,144::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:07:06,205::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:07:06,205::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:07:06,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:07:06,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:06,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:07:06,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:07:06,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:07:06,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:06,208::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438026|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438026 (Thu Nov 14 16:07:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:07:06,208::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:07:06,216::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:07:06,216::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:07:06,216::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438026, 'last-update-local-ts': 1384438026.21595, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:06,216::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:07:06,216::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438023, 'last-update-local-ts': 1384438026.21595, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:06,217::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:07:16,289::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:07:16,545::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:07:16,617::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:07:16,678::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:07:16,678::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:07:16,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:07:16,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:16,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:07:16,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:07:16,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 16:07:16,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:16,680::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438036|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438036 (Thu Nov 14 16:07:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:07:16,681::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:07:16,689::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:07:16,690::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:07:16,690::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438036, 'last-update-local-ts': 1384438036.689575, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:16,690::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:07:16,690::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438033, 'last-update-local-ts': 1384438036.689575, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:16,690::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:07:26,775::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:07:27,024::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:07:27,103::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:07:27,163::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:07:27,164::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:07:27,164::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:07:27,164::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:27,165::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:07:27,165::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:07:27,165::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 16:07:27,165::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:27,166::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438047|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438047 (Thu Nov 14 16:07:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:07:27,166::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:07:27,174::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:07:27,174::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:07:27,174::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438047, 'last-update-local-ts': 1384438047.1742549, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:27,175::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:07:27,175::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438044, 'last-update-local-ts': 1384438047.1742549, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:27,175::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:07:37,251::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:07:37,529::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:07:37,627::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:07:37,688::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:07:37,688::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:07:37,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:07:37,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:37,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:07:37,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:07:37,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 16:07:37,690::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:37,691::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438057|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438057 (Thu Nov 14 16:07:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:07:37,691::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:07:37,700::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:07:37,700::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:07:37,701::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438057, 'last-update-local-ts': 1384438057.7004409, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:37,701::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:07:37,701::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438054, 'last-update-local-ts': 1384438057.7004409, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:37,701::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:07:47,777::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:07:48,035::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:07:48,123::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:07:48,183::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:07:48,183::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:07:48,184::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:07:48,184::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:48,184::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:07:48,185::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:07:48,185::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 16:07:48,185::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:48,185::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438068|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438068 (Thu Nov 14 16:07:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:07:48,186::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:07:48,194::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:07:48,194::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:07:48,194::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438068, 'last-update-local-ts': 1384438068.194309, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:48,195::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:07:48,195::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438065, 'last-update-local-ts': 1384438068.194309, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:48,195::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:07:58,270::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:07:58,524::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:07:58,605::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:07:58,671::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:07:58,671::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:07:58,677::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:07:58,678::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:58,678::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:07:58,678::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:07:58,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 16:07:58,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:07:58,680::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438078|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438078 (Thu Nov 14 16:07:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:07:58,680::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:07:58,690::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:07:58,691::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:07:58,691::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438078, 'last-update-local-ts': 1384438078.690522, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:58,691::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:07:58,691::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438075, 'last-update-local-ts': 1384438078.690522, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:07:58,691::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:08:08,767::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:08:09,015::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:08:09,069::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:08:09,128::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:08:09,129::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:08:09,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:08:09,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:09,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:08:09,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:08:09,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:08:09,130::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:09,131::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438089|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438089 (Thu Nov 14 16:08:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:08:09,131::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:08:09,139::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:08:09,140::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:08:09,140::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438089, 'last-update-local-ts': 1384438089.13975, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:09,140::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:08:09,140::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438086, 'last-update-local-ts': 1384438089.13975, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:09,141::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:08:19,217::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:08:19,472::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:08:19,561::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:08:19,634::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:08:19,634::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:08:19,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:08:19,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:19,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:08:19,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:08:19,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.63 >MainThread::INFO::2013-11-14 16:08:19,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:19,636::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438099|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438099 (Thu Nov 14 16:08:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.63 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:08:19,637::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:08:19,645::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:08:19,645::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:08:19,646::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438099, 'last-update-local-ts': 1384438099.6454799, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:19,646::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:08:19,646::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438096, 'last-update-local-ts': 1384438099.6454799, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:19,646::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:08:29,720::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:08:29,977::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:08:30,095::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:08:30,156::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:08:30,157::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:08:30,157::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:08:30,157::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:30,158::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:08:30,158::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:08:30,158::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.45 >MainThread::INFO::2013-11-14 16:08:30,158::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:30,160::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438110|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438110 (Thu Nov 14 16:08:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.45 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:08:30,160::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:08:30,172::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:08:30,172::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:08:30,173::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438110, 'last-update-local-ts': 1384438110.1724191, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:30,173::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:08:30,173::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438107, 'last-update-local-ts': 1384438110.1724191, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:30,173::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:08:40,250::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:08:40,507::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:08:40,590::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:08:40,662::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:08:40,662::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:08:40,663::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:08:40,663::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:40,663::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:08:40,664::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:08:40,664::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.385 >MainThread::INFO::2013-11-14 16:08:40,664::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:40,665::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438120|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438120 (Thu Nov 14 16:08:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.385 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:08:40,665::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:08:40,674::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:08:40,674::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:08:40,674::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438120, 'last-update-local-ts': 1384438120.6738949, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:40,674::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:08:40,674::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438117, 'last-update-local-ts': 1384438120.6738949, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:40,675::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:08:50,756::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:08:51,025::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:08:51,107::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:08:51,168::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:08:51,168::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:08:51,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:08:51,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:51,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:08:51,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:08:51,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.325 >MainThread::INFO::2013-11-14 16:08:51,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:08:51,171::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438131|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438131 (Thu Nov 14 16:08:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.325 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:08:51,172::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:08:51,180::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:08:51,180::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:08:51,181::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438131, 'last-update-local-ts': 1384438131.1804609, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:51,181::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:08:51,181::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438128, 'last-update-local-ts': 1384438131.1804609, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:08:51,181::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:09:01,257::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:09:01,516::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:09:01,578::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:09:01,643::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:09:01,643::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:09:01,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:09:01,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:01,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:09:01,645::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:09:01,645::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.275 >MainThread::INFO::2013-11-14 16:09:01,645::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:01,646::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438141|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438141 (Thu Nov 14 16:09:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.275 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:09:01,646::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:09:01,654::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:09:01,655::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:09:01,655::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438141, 'last-update-local-ts': 1384438141.6545801, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:01,655::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:09:01,655::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438138, 'last-update-local-ts': 1384438141.6545801, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:01,655::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:09:11,728::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:09:11,982::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:09:12,070::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:09:12,135::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:09:12,136::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:09:12,136::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:09:12,136::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:12,137::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:09:12,137::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:09:12,137::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 16:09:12,138::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:12,138::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438152|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438152 (Thu Nov 14 16:09:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:09:12,138::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:09:12,147::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:09:12,147::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:09:12,147::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438152, 'last-update-local-ts': 1384438152.1469729, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:12,147::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:09:12,148::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438148, 'last-update-local-ts': 1384438152.1469729, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:12,148::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:09:22,234::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:09:22,490::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:09:22,582::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:09:22,644::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:09:22,644::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:09:22,645::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:09:22,645::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:22,645::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:09:22,646::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:09:22,646::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 16:09:22,646::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:22,647::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438162|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438162 (Thu Nov 14 16:09:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:09:22,648::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:09:22,654::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:09:22,654::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:09:22,655::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438162, 'last-update-local-ts': 1384438162.6544001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:22,655::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:09:22,655::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438159, 'last-update-local-ts': 1384438162.6544001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:22,655::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:09:32,724::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:09:32,997::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:09:33,084::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:09:33,145::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:09:33,145::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:09:33,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:09:33,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:33,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:09:33,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:09:33,147::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.195 >MainThread::INFO::2013-11-14 16:09:33,147::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:33,147::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438173|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438173 (Thu Nov 14 16:09:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.195 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:09:33,148::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:09:33,154::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:09:33,155::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:09:33,155::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438173, 'last-update-local-ts': 1384438173.1545949, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:33,155::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:09:33,155::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438169, 'last-update-local-ts': 1384438173.1545949, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:33,155::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:09:43,229::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:09:43,482::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:09:43,551::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:09:43,610::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:09:43,610::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:09:43,611::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:09:43,611::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:43,611::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:09:43,611::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:09:43,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.165 >MainThread::INFO::2013-11-14 16:09:43,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:43,612::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438183|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438183 (Thu Nov 14 16:09:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.165 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:09:43,613::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:09:43,621::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:09:43,621::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:09:43,621::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438183, 'last-update-local-ts': 1384438183.6211231, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:43,621::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:09:43,622::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438180, 'last-update-local-ts': 1384438183.6211231, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:43,622::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:09:53,719::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:09:53,966::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:09:54,060::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:09:54,124::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:09:54,125::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:09:54,125::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:09:54,125::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:54,126::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:09:54,126::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:09:54,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 16:09:54,127::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:09:54,128::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438194|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438194 (Thu Nov 14 16:09:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:09:54,128::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:09:54,138::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:09:54,139::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:09:54,139::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438194, 'last-update-local-ts': 1384438194.1387861, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:54,139::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:09:54,139::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438190, 'last-update-local-ts': 1384438194.1387861, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:09:54,140::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:10:04,229::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:10:04,489::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:10:04,628::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:10:04,695::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:10:04,695::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:10:04,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:10:04,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:04,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:10:04,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:10:04,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 16:10:04,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:04,698::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438204|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438204 (Thu Nov 14 16:10:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:10:04,699::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:10:04,707::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:10:04,708::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:10:04,708::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438204, 'last-update-local-ts': 1384438204.7076011, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:04,708::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:10:04,708::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438201, 'last-update-local-ts': 1384438204.7076011, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:04,708::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:10:14,784::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:10:15,038::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:10:15,126::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:10:15,187::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:10:15,187::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:10:15,188::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:10:15,188::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:15,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:10:15,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:10:15,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 16:10:15,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:15,190::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438215|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438215 (Thu Nov 14 16:10:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:10:15,190::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:10:15,198::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:10:15,198::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:10:15,199::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438215, 'last-update-local-ts': 1384438215.1985421, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:15,199::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:10:15,199::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438211, 'last-update-local-ts': 1384438215.1985421, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:15,199::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:10:25,273::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:10:25,535::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:10:25,624::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:10:25,684::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:10:25,684::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:10:25,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:10:25,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:25,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:10:25,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:10:25,686::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 16:10:25,686::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:25,687::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438225|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438225 (Thu Nov 14 16:10:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:10:25,687::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:10:25,697::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:10:25,697::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:10:25,698::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438225, 'last-update-local-ts': 1384438225.6973031, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:25,698::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:10:25,698::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438222, 'last-update-local-ts': 1384438225.6973031, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:25,698::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:10:35,776::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:10:36,042::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:10:36,095::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:10:36,154::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:10:36,154::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:10:36,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:10:36,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:36,156::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:10:36,156::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:10:36,156::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 16:10:36,156::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:36,157::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438236|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438236 (Thu Nov 14 16:10:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:10:36,157::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:10:36,164::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:10:36,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:10:36,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438236, 'last-update-local-ts': 1384438236.1640201, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:36,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:10:36,165::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438232, 'last-update-local-ts': 1384438236.1640201, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:36,165::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:10:46,241::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:10:46,495::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:10:46,583::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:10:46,642::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:10:46,642::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:10:46,643::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:10:46,643::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:46,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:10:46,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:10:46,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 16:10:46,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:46,645::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438246|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438246 (Thu Nov 14 16:10:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:10:46,645::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:10:46,653::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:10:46,654::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:10:46,654::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438246, 'last-update-local-ts': 1384438246.653784, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:46,654::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:10:46,654::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438243, 'last-update-local-ts': 1384438246.653784, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:46,655::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:10:56,726::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:10:56,977::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:10:57,070::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:10:57,134::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:10:57,134::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:10:57,135::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:10:57,135::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:57,135::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:10:57,136::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:10:57,136::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 16:10:57,136::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:10:57,137::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438257|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438257 (Thu Nov 14 16:10:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:10:57,138::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:10:57,147::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:10:57,147::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:10:57,147::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438257, 'last-update-local-ts': 1384438257.1470389, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:57,147::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:10:57,148::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438253, 'last-update-local-ts': 1384438257.1470389, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:10:57,148::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:11:07,230::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:11:07,548::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:11:07,640::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:11:07,701::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:11:07,702::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:11:07,702::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:11:07,702::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:07,703::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:11:07,703::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:11:07,703::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 16:11:07,703::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:07,704::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438267|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438267 (Thu Nov 14 16:11:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:11:07,704::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:11:07,712::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:11:07,713::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:11:07,713::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438267, 'last-update-local-ts': 1384438267.712805, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:07,713::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:11:07,713::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438264, 'last-update-local-ts': 1384438267.712805, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:07,714::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:11:17,784::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:11:18,039::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:11:18,104::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:11:18,164::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:11:18,164::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:11:18,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:11:18,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:18,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:11:18,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:11:18,172::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 16:11:18,172::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:18,172::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438278|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438278 (Thu Nov 14 16:11:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:11:18,173::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:11:18,183::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:11:18,183::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:11:18,183::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438278, 'last-update-local-ts': 1384438278.1831131, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:18,183::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:11:18,184::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438274, 'last-update-local-ts': 1384438278.1831131, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:18,184::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:11:28,267::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:11:28,521::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:11:28,654::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:11:28,727::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:11:28,728::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:11:28,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:11:28,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:28,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:11:28,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:11:28,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 16:11:28,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:28,731::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438288|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438288 (Thu Nov 14 16:11:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:11:28,731::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:11:28,740::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:11:28,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:11:28,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438288, 'last-update-local-ts': 1384438288.7401519, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:28,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:11:28,741::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438285, 'last-update-local-ts': 1384438288.7401519, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:28,741::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:11:38,819::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:11:39,066::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:11:39,141::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:11:39,209::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:11:39,209::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:11:39,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:11:39,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:39,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:11:39,210::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:11:39,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 16:11:39,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:39,211::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438299|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438299 (Thu Nov 14 16:11:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:11:39,211::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:11:39,220::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:11:39,220::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:11:39,220::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438299, 'last-update-local-ts': 1384438299.22034, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:39,221::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:11:39,221::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438295, 'last-update-local-ts': 1384438299.22034, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:39,221::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:11:49,308::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:11:49,564::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:11:49,647::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:11:49,708::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:11:49,708::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:11:49,708::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:11:49,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:49,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:11:49,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:11:49,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 16:11:49,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:11:49,711::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438309|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438309 (Thu Nov 14 16:11:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:11:49,711::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:11:49,721::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:11:49,721::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:11:49,721::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438309, 'last-update-local-ts': 1384438309.7209401, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:49,721::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:11:49,722::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438305, 'last-update-local-ts': 1384438309.7209401, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:11:49,722::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:11:59,807::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:12:00,061::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:12:00,137::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:12:00,206::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:12:00,206::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:12:00,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:12:00,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:00,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:12:00,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:12:00,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 16:12:00,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:00,208::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438320|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438320 (Thu Nov 14 16:12:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:12:00,208::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:12:00,217::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:12:00,217::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:12:00,218::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438320, 'last-update-local-ts': 1384438320.2174809, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:00,218::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:12:00,218::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438316, 'last-update-local-ts': 1384438320.2174809, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:00,218::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:12:10,304::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:12:10,558::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:12:10,626::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:12:10,687::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:12:10,687::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:12:10,688::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:12:10,688::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:10,688::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:12:10,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:12:10,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 16:12:10,689::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:10,690::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438330|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438330 (Thu Nov 14 16:12:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:12:10,690::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:12:10,698::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:12:10,698::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:12:10,699::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438330, 'last-update-local-ts': 1384438330.6983621, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:10,699::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:12:10,699::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438326, 'last-update-local-ts': 1384438330.6983621, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:10,699::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:12:20,773::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:12:21,025::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:12:21,118::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:12:21,177::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:12:21,178::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:12:21,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:12:21,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:21,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:12:21,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:12:21,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 16:12:21,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:21,181::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438341|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438341 (Thu Nov 14 16:12:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:12:21,181::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:12:21,190::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:12:21,190::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:12:21,190::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438341, 'last-update-local-ts': 1384438341.1902239, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:21,191::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:12:21,191::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438337, 'last-update-local-ts': 1384438341.1902239, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:21,191::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:12:31,281::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:12:31,534::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:12:31,614::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:12:31,680::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:12:31,681::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:12:31,681::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:12:31,681::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:31,682::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:12:31,682::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:12:31,682::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.01 >MainThread::INFO::2013-11-14 16:12:31,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:31,683::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438351|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438351 (Thu Nov 14 16:12:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.01 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:12:31,683::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:12:31,692::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:12:31,692::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:12:31,692::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438351, 'last-update-local-ts': 1384438351.6920021, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:31,692::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:12:31,693::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438347, 'last-update-local-ts': 1384438351.6920021, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:31,693::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:12:41,778::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:12:42,031::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:12:42,107::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:12:42,167::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:12:42,168::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:12:42,174::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:12:42,174::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:42,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:12:42,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:12:42,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.01 >MainThread::INFO::2013-11-14 16:12:42,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:42,176::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438362|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438362 (Thu Nov 14 16:12:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.01 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:12:42,176::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:12:42,186::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:12:42,187::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:12:42,187::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438362, 'last-update-local-ts': 1384438362.186528, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:42,187::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:12:42,187::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438358, 'last-update-local-ts': 1384438362.186528, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:42,187::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:12:52,260::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:12:52,510::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:12:52,578::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:12:52,651::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:12:52,651::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:12:52,652::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:12:52,652::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:52,653::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:12:52,653::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:12:52,653::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:12:52,654::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:12:52,655::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438372|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438372 (Thu Nov 14 16:12:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:12:52,655::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:12:52,664::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:12:52,665::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:12:52,665::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438372, 'last-update-local-ts': 1384438372.6645291, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:52,665::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:12:52,665::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438368, 'last-update-local-ts': 1384438372.6645291, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:12:52,665::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:13:02,784::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:13:03,174::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:13:03,249::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:13:03,318::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:13:03,319::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:13:03,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:13:03,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:03,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 16:13:03,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:13:03,321::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 16:13:03,321::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:03,321::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438383|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438383 (Thu Nov 14 16:13:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:13:03,321::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:13:03,329::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:13:03,330::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:13:03,330::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438383, 'last-update-local-ts': 1384438383.3296001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:03,330::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:13:03,330::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438379, 'last-update-local-ts': 1384438383.3296001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:03,331::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:13:13,401::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:13:13,647::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:13:13,725::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:13:13,831::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:13:13,831::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:13:13,832::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:13:13,832::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:13,832::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:13:13,833::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:13:13,833::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 16:13:13,833::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:13,833::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438393|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438393 (Thu Nov 14 16:13:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:13:13,834::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:13:13,842::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:13:13,842::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:13:13,842::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438393, 'last-update-local-ts': 1384438393.842212, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:13,843::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:13:13,843::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438389, 'last-update-local-ts': 1384438393.842212, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:13,843::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:13:23,944::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:13:24,198::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:13:24,288::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:13:24,349::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:13:24,349::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:13:24,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:13:24,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:24,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:13:24,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:13:24,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 16:13:24,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:24,352::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438404|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438404 (Thu Nov 14 16:13:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:13:24,352::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:13:24,360::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:13:24,361::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:13:24,361::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438404, 'last-update-local-ts': 1384438404.360641, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:24,361::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:13:24,361::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438400, 'last-update-local-ts': 1384438404.360641, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:24,361::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:13:34,455::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:13:34,707::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:13:34,789::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:13:34,857::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:13:34,857::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:13:34,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:13:34,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:34,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:13:34,859::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:13:34,859::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:13:34,859::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:34,860::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438414|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438414 (Thu Nov 14 16:13:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:13:34,860::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:13:34,869::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:13:34,869::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:13:34,869::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438414, 'last-update-local-ts': 1384438414.868995, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:34,869::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:13:34,870::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438410, 'last-update-local-ts': 1384438414.868995, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:34,870::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:13:44,950::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:13:45,208::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:13:45,292::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:13:45,352::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:13:45,353::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:13:45,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:13:45,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:45,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:13:45,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:13:45,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 16:13:45,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:45,361::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438425|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438425 (Thu Nov 14 16:13:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:13:45,361::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:13:45,369::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:13:45,370::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:13:45,370::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438425, 'last-update-local-ts': 1384438425.369534, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:45,370::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:13:45,370::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438420, 'last-update-local-ts': 1384438425.369534, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:45,370::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:13:55,436::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:13:55,695::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:13:55,751::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:13:55,810::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:13:55,810::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:13:55,811::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:13:55,811::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:55,811::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:13:55,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:13:55,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 16:13:55,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:13:55,813::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438435|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438435 (Thu Nov 14 16:13:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:13:55,813::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:13:55,823::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:13:55,823::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:13:55,823::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438435, 'last-update-local-ts': 1384438435.823169, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:55,824::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:13:55,824::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438431, 'last-update-local-ts': 1384438435.823169, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:13:55,824::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:14:05,909::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:14:06,160::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:14:06,262::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:14:06,367::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:14:06,368::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:14:06,368::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:14:06,368::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:06,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7044 >MainThread::INFO::2013-11-14 16:14:06,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:14:06,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 16:14:06,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:06,370::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438446|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438446 (Thu Nov 14 16:14:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7044 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:14:06,371::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:14:06,381::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:14:06,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:14:06,382::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438446, 'last-update-local-ts': 1384438446.381361, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:06,382::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:14:06,382::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438441, 'last-update-local-ts': 1384438446.381361, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:06,382::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:14:16,465::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:14:16,713::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:14:16,798::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:14:16,858::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:14:16,859::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:14:16,865::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:14:16,866::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:16,866::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7043 >MainThread::INFO::2013-11-14 16:14:16,866::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:14:16,866::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 16:14:16,867::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:16,867::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438456|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438456 (Thu Nov 14 16:14:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7043 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:14:16,867::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:14:16,878::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:14:16,879::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:14:16,879::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438456, 'last-update-local-ts': 1384438456.8785501, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:16,879::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:14:16,880::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438452, 'last-update-local-ts': 1384438456.8785501, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:16,880::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:14:26,979::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:14:27,254::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:14:27,383::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:14:27,463::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:14:27,464::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:14:27,464::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:14:27,465::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:27,465::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7027 >MainThread::INFO::2013-11-14 16:14:27,465::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:14:27,466::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 16:14:27,466::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:27,467::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438467|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438467 (Thu Nov 14 16:14:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7027 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:14:27,467::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:14:27,487::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:14:27,488::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:14:27,488::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438467, 'last-update-local-ts': 1384438467.4874971, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:27,488::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:14:27,488::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438462, 'last-update-local-ts': 1384438467.4874971, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:27,488::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:14:37,561::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:14:37,837::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:14:37,939::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:14:38,000::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:14:38,000::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:14:38,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0015243902439 >MainThread::INFO::2013-11-14 16:14:38,001::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:38,002::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7000 >MainThread::INFO::2013-11-14 16:14:38,002::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:14:38,002::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 16:14:38,002::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:38,003::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438478|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438478 (Thu Nov 14 16:14:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7000 >mem-load=0.0015243902439 > >MainThread::INFO::2013-11-14 16:14:38,004::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:14:38,010::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:14:38,011::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:14:38,011::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438478, 'last-update-local-ts': 1384438478.0107901, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:38,011::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:14:38,011::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438473, 'last-update-local-ts': 1384438478.0107901, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:38,012::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:14:48,087::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:14:48,354::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:14:48,433::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:14:48,524::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:14:48,524::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:14:48,524::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:14:48,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:48,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:14:48,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:14:48,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 16:14:48,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:48,530::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438488|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438488 (Thu Nov 14 16:14:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:14:48,530::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:14:48,547::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:14:48,552::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:14:48,553::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438488, 'last-update-local-ts': 1384438488.546988, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:48,553::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:14:48,553::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438483, 'last-update-local-ts': 1384438488.546988, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:48,553::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:14:58,665::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:14:58,952::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:14:59,007::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:14:59,069::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:14:59,069::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:14:59,069::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:14:59,070::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:59,070::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:14:59,070::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:14:59,071::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:14:59,071::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:14:59,072::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438499|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438499 (Thu Nov 14 16:14:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:14:59,072::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:14:59,084::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:14:59,084::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:14:59,084::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438499, 'last-update-local-ts': 1384438499.083869, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:59,084::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:14:59,085::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438494, 'last-update-local-ts': 1384438499.083869, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:14:59,085::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:15:09,166::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:15:09,414::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:15:09,484::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:15:09,544::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:15:09,544::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:15:09,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:15:09,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:09,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:15:09,545::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:15:09,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 16:15:09,546::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:09,547::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438509|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438509 (Thu Nov 14 16:15:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:15:09,547::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:15:09,556::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:15:09,556::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:15:09,556::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438509, 'last-update-local-ts': 1384438509.556252, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:09,557::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:15:09,557::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438504, 'last-update-local-ts': 1384438509.556252, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:09,557::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:15:19,649::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:15:19,899::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:15:19,972::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:15:20,039::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:15:20,039::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:15:20,040::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:15:20,040::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:20,040::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:15:20,041::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:15:20,041::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 16:15:20,041::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:20,041::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438520|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438520 (Thu Nov 14 16:15:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:15:20,042::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:15:20,050::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:15:20,051::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:15:20,051::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438520, 'last-update-local-ts': 1384438520.05059, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:20,051::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:15:20,051::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438515, 'last-update-local-ts': 1384438520.05059, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:20,051::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:15:30,138::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:15:30,418::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:15:30,502::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:15:30,563::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:15:30,563::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:15:30,570::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:15:30,570::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:30,570::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:15:30,571::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:15:30,571::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 16:15:30,571::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:30,572::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438530|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438530 (Thu Nov 14 16:15:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:15:30,572::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:15:30,581::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:15:30,582::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:15:30,582::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438530, 'last-update-local-ts': 1384438530.5818031, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:30,582::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:15:30,582::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438525, 'last-update-local-ts': 1384438530.5818031, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:30,583::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:15:40,679::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:15:40,929::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:15:41,002::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:15:41,063::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:15:41,063::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:15:41,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:15:41,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:41,064::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:15:41,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:15:41,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 16:15:41,065::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:41,066::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438541|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438541 (Thu Nov 14 16:15:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:15:41,066::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:15:41,075::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:15:41,075::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:15:41,075::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438541, 'last-update-local-ts': 1384438541.0751081, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:41,075::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:15:41,076::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438535, 'last-update-local-ts': 1384438541.0751081, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:41,076::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:15:51,146::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:15:51,404::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:15:51,479::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:15:51,538::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:15:51,539::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:15:51,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:15:51,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:51,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:15:51,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:15:51,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 16:15:51,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:15:51,541::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438551|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438551 (Thu Nov 14 16:15:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:15:51,541::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:15:51,550::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:15:51,550::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:15:51,550::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438551, 'last-update-local-ts': 1384438551.5499401, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:51,550::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:15:51,550::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438546, 'last-update-local-ts': 1384438551.5499401, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:15:51,551::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:16:01,630::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:16:01,897::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:16:02,002::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:16:02,065::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:16:02,065::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:16:02,066::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:16:02,066::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:02,067::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:16:02,067::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:16:02,067::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.01 >MainThread::INFO::2013-11-14 16:16:02,068::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:02,069::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438562|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438562 (Thu Nov 14 16:16:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.01 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:16:02,069::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:16:02,078::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:16:02,078::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:16:02,079::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438562, 'last-update-local-ts': 1384438562.0783579, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:02,079::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:16:02,079::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438556, 'last-update-local-ts': 1384438562.0783579, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:02,079::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:16:12,176::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:16:12,424::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:16:12,508::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:16:12,568::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:16:12,569::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:16:12,569::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:16:12,570::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:12,570::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:16:12,570::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:16:12,570::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 16:16:12,571::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:12,571::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438572|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438572 (Thu Nov 14 16:16:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:16:12,571::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:16:12,580::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:16:12,580::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:16:12,581::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438572, 'last-update-local-ts': 1384438572.580477, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:12,581::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:16:12,581::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438567, 'last-update-local-ts': 1384438572.580477, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:12,581::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:16:22,663::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:16:22,919::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:16:23,019::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:16:23,083::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:16:23,083::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:16:23,084::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:16:23,084::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:23,084::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:16:23,084::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:16:23,085::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 16:16:23,085::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:23,085::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438583|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438583 (Thu Nov 14 16:16:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:16:23,086::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:16:23,092::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:16:23,092::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:16:23,093::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438583, 'last-update-local-ts': 1384438583.092514, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:23,093::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:16:23,093::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438577, 'last-update-local-ts': 1384438583.092514, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:23,093::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:16:33,182::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:16:33,447::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:16:33,515::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:16:33,574::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:16:33,574::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:16:33,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:16:33,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:33,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:16:33,575::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:16:33,576::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 16:16:33,576::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:33,577::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438593|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438593 (Thu Nov 14 16:16:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:16:33,577::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:16:33,584::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:16:33,584::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:16:33,585::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438593, 'last-update-local-ts': 1384438593.5844049, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:33,585::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:16:33,585::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438588, 'last-update-local-ts': 1384438593.5844049, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:33,585::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:16:43,682::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:16:43,931::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:16:44,006::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:16:44,068::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:16:44,068::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:16:44,069::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:16:44,069::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:44,069::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:16:44,069::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:16:44,070::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 16:16:44,070::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:44,070::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438604|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438604 (Thu Nov 14 16:16:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:16:44,071::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:16:44,079::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:16:44,079::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:16:44,080::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438604, 'last-update-local-ts': 1384438604.079535, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:44,080::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:16:44,080::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438598, 'last-update-local-ts': 1384438604.079535, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:44,080::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:16:54,168::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:16:54,421::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:16:54,514::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:16:54,575::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:16:54,576::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:16:54,582::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:16:54,583::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:54,583::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:16:54,583::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:16:54,584::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 16:16:54,584::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:16:54,584::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438614|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438614 (Thu Nov 14 16:16:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:16:54,584::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:16:54,594::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:16:54,595::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:16:54,595::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438614, 'last-update-local-ts': 1384438614.5948629, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:54,595::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:16:54,595::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438609, 'last-update-local-ts': 1384438614.5948629, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:16:54,596::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:17:04,678::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:17:04,934::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:17:05,033::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:17:05,093::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:17:05,093::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:17:05,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:17:05,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:05,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:17:05,094::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:17:05,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 16:17:05,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:05,096::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438625|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438625 (Thu Nov 14 16:17:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:17:05,096::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:17:05,105::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:17:05,105::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:17:05,105::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438625, 'last-update-local-ts': 1384438625.104964, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:05,105::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:17:05,105::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438619, 'last-update-local-ts': 1384438625.104964, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:05,106::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:17:15,214::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:17:15,531::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:17:15,624::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:17:15,685::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:17:15,685::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:17:15,692::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:17:15,692::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:15,692::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:17:15,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:17:15,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 16:17:15,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:15,694::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438635|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438635 (Thu Nov 14 16:17:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:17:15,694::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:17:15,703::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:17:15,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:17:15,704::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438635, 'last-update-local-ts': 1384438635.7035689, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:15,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:17:15,704::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438630, 'last-update-local-ts': 1384438635.7035689, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:15,704::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:17:25,794::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:17:26,049::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:17:26,110::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:17:26,169::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:17:26,169::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:17:26,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:17:26,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:26,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:17:26,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:17:26,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:17:26,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:26,172::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438646|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438646 (Thu Nov 14 16:17:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:17:26,172::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:17:26,181::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:17:26,181::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:17:26,182::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438646, 'last-update-local-ts': 1384438646.1814089, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:26,182::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:17:26,182::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438640, 'last-update-local-ts': 1384438646.1814089, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:26,182::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:17:36,284::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:17:36,532::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:17:36,621::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:17:36,686::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:17:36,686::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:17:36,687::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:17:36,687::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:36,687::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:17:36,687::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:17:36,688::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 16:17:36,688::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:36,688::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438656|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438656 (Thu Nov 14 16:17:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:17:36,689::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:17:36,697::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:17:36,697::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:17:36,698::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438656, 'last-update-local-ts': 1384438656.697443, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:36,698::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:17:36,698::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438650, 'last-update-local-ts': 1384438656.697443, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:36,698::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:17:46,804::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:17:47,060::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:17:47,145::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:17:47,251::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:17:47,251::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:17:47,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:17:47,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:47,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:17:47,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:17:47,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 16:17:47,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:47,260::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438667|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438667 (Thu Nov 14 16:17:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:17:47,260::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:17:47,270::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:17:47,270::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:17:47,270::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438667, 'last-update-local-ts': 1384438667.270221, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:47,271::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:17:47,271::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438661, 'last-update-local-ts': 1384438667.270221, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:47,271::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:17:57,352::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:17:57,605::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:17:57,684::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:17:57,743::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:17:57,743::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:17:57,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:17:57,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:57,744::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:17:57,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:17:57,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 16:17:57,745::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:17:57,746::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438677|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438677 (Thu Nov 14 16:17:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:17:57,746::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:17:57,756::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:17:57,756::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:17:57,756::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438677, 'last-update-local-ts': 1384438677.755959, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:57,756::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:17:57,756::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438671, 'last-update-local-ts': 1384438677.755959, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:17:57,757::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:18:07,840::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:18:08,114::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:18:08,194::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:18:08,255::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:18:08,255::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:18:08,256::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:18:08,256::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:08,256::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:18:08,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:18:08,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:18:08,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:08,258::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438688|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438688 (Thu Nov 14 16:18:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:18:08,258::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:18:08,266::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:18:08,267::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:18:08,267::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438688, 'last-update-local-ts': 1384438688.2666731, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:08,267::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:18:08,267::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438682, 'last-update-local-ts': 1384438688.2666731, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:08,267::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:18:18,351::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:18:18,600::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:18:18,679::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:18:18,739::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:18:18,739::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:18:18,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:18:18,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:18,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:18:18,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:18:18,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 16:18:18,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:18,748::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438698|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438698 (Thu Nov 14 16:18:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:18:18,748::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:18:18,758::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:18:18,758::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:18:18,759::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438698, 'last-update-local-ts': 1384438698.758414, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:18,759::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:18:18,759::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438692, 'last-update-local-ts': 1384438698.758414, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:18,759::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:18:28,844::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:18:29,098::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:18:29,196::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:18:29,257::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:18:29,257::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:18:29,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:18:29,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:29,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:18:29,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:18:29,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 16:18:29,259::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:29,260::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438709|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438709 (Thu Nov 14 16:18:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:18:29,261::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:18:29,271::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:18:29,271::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:18:29,271::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438709, 'last-update-local-ts': 1384438709.2710979, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:29,271::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:18:29,272::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438703, 'last-update-local-ts': 1384438709.2710979, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:29,272::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:18:39,347::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:18:39,606::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:18:39,690::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:18:39,755::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:18:39,756::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:18:39,756::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:18:39,756::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:39,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:18:39,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:18:39,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.52 >MainThread::INFO::2013-11-14 16:18:39,757::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:39,758::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438719|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438719 (Thu Nov 14 16:18:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.52 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:18:39,758::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:18:39,766::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:18:39,766::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:18:39,767::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438719, 'last-update-local-ts': 1384438719.7665091, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:39,767::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:18:39,767::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438713, 'last-update-local-ts': 1384438719.7665091, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:39,767::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:18:49,833::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:18:50,090::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:18:50,145::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:18:50,204::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:18:50,204::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:18:50,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:18:50,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:50,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:18:50,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:18:50,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.44 >MainThread::INFO::2013-11-14 16:18:50,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:18:50,206::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438730|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438730 (Thu Nov 14 16:18:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.44 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:18:50,207::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:18:50,216::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:18:50,216::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:18:50,217::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438730, 'last-update-local-ts': 1384438730.216466, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:50,217::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:18:50,217::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438724, 'last-update-local-ts': 1384438730.216466, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:18:50,217::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:19:00,305::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:19:00,582::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:19:00,731::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:19:00,793::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:19:00,793::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:19:00,793::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:19:00,794::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:00,794::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:19:00,794::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:19:00,794::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.315 >MainThread::INFO::2013-11-14 16:19:00,795::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:00,796::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438740|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438740 (Thu Nov 14 16:19:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.315 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:19:00,796::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:19:00,805::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:19:00,805::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:19:00,805::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438740, 'last-update-local-ts': 1384438740.8049569, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:00,805::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:19:00,805::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438734, 'last-update-local-ts': 1384438740.8049569, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:00,806::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:19:10,882::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:19:11,135::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:19:11,222::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:19:11,281::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:19:11,282::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:19:11,282::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:19:11,282::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:11,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:19:11,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:19:11,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 16:19:11,284::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:11,284::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438751|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438751 (Thu Nov 14 16:19:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.265 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:19:11,284::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:19:11,293::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:19:11,293::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:19:11,293::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438751, 'last-update-local-ts': 1384438751.2932439, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:11,294::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:19:11,294::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438745, 'last-update-local-ts': 1384438751.2932439, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:11,294::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:19:21,366::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:19:21,619::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:19:21,713::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:19:21,774::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:19:21,774::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:19:21,775::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:19:21,775::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:21,775::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 16:19:21,775::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:19:21,776::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 16:19:21,776::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:21,777::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438761|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438761 (Thu Nov 14 16:19:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:19:21,777::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:19:21,786::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:19:21,787::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:19:21,787::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438761, 'last-update-local-ts': 1384438761.7866831, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:21,787::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:19:21,787::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438755, 'last-update-local-ts': 1384438761.7866831, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:21,788::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:19:31,866::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:19:32,119::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:19:32,204::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:19:32,265::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:19:32,266::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:19:32,266::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:19:32,266::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:32,267::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:19:32,267::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:19:32,267::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 16:19:32,267::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:32,268::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438772|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438772 (Thu Nov 14 16:19:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:19:32,268::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:19:32,276::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:19:32,277::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:19:32,277::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438772, 'last-update-local-ts': 1384438772.276803, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:32,277::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:19:32,277::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438766, 'last-update-local-ts': 1384438772.276803, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:32,278::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:19:42,344::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:19:42,591::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:19:42,650::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:19:42,709::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:19:42,710::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:19:42,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:19:42,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:42,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:19:42,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:19:42,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.195 >MainThread::INFO::2013-11-14 16:19:42,718::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:42,718::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438782|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438782 (Thu Nov 14 16:19:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.195 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:19:42,718::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:19:42,727::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:19:42,728::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:19:42,728::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438782, 'last-update-local-ts': 1384438782.727824, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:42,728::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:19:42,728::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438776, 'last-update-local-ts': 1384438782.727824, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:42,729::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:19:52,810::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:19:53,103::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:19:53,209::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:19:53,271::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:19:53,271::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:19:53,272::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:19:53,272::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:53,272::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:19:53,273::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:19:53,273::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.165 >MainThread::INFO::2013-11-14 16:19:53,273::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:19:53,274::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438793|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438793 (Thu Nov 14 16:19:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.165 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:19:53,274::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:19:53,281::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:19:53,287::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:19:53,288::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438793, 'last-update-local-ts': 1384438793.281301, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:53,288::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:19:53,288::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438786, 'last-update-local-ts': 1384438793.281301, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:19:53,289::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:20:03,407::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:20:03,682::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:20:03,810::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:20:03,871::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:20:03,871::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:20:03,871::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:20:03,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:03,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:20:03,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:20:03,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 16:20:03,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:03,874::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438803|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438803 (Thu Nov 14 16:20:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:20:03,874::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:20:03,881::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:20:03,881::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:20:03,881::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438803, 'last-update-local-ts': 1384438803.881037, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:03,881::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:20:03,882::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438797, 'last-update-local-ts': 1384438803.881037, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:03,882::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:20:13,985::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:20:14,235::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:20:14,326::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:20:14,394::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:20:14,394::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:20:14,395::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:20:14,395::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:14,395::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:20:14,396::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:20:14,396::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:20:14,396::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:14,396::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438814|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438814 (Thu Nov 14 16:20:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:20:14,397::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:20:14,405::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:20:14,405::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:20:14,406::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438814, 'last-update-local-ts': 1384438814.4053781, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:14,406::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:20:14,406::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438807, 'last-update-local-ts': 1384438814.4053781, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:14,406::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:20:24,495::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:20:24,747::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:20:24,837::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:20:24,897::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:20:24,897::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:20:24,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:20:24,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:24,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:20:24,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:20:24,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 16:20:24,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:24,899::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438824|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438824 (Thu Nov 14 16:20:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:20:24,900::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:20:24,907::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:20:24,907::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:20:24,907::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438824, 'last-update-local-ts': 1384438824.907016, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:24,907::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:20:24,908::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438818, 'last-update-local-ts': 1384438824.907016, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:24,908::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:20:34,980::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:20:35,233::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:20:35,295::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:20:35,356::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:20:35,356::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:20:35,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:20:35,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:35,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:20:35,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:20:35,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 16:20:35,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:35,359::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438835|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438835 (Thu Nov 14 16:20:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:20:35,359::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:20:35,366::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:20:35,366::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:20:35,367::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438835, 'last-update-local-ts': 1384438835.3663819, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:35,367::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:20:35,367::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438828, 'last-update-local-ts': 1384438835.3663819, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:35,367::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:20:45,453::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:20:45,709::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:20:45,782::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:20:45,847::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:20:45,848::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:20:45,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:20:45,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:45,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:20:45,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:20:45,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 16:20:45,850::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:45,850::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438845|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438845 (Thu Nov 14 16:20:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:20:45,850::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:20:45,857::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:20:45,857::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:20:45,857::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438845, 'last-update-local-ts': 1384438845.85709, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:45,857::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:20:45,858::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438839, 'last-update-local-ts': 1384438845.85709, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:45,858::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:20:55,936::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:20:56,188::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:20:56,278::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:20:56,338::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:20:56,339::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:20:56,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:20:56,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:56,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:20:56,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:20:56,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:20:56,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:20:56,341::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438856|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438856 (Thu Nov 14 16:20:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:20:56,341::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:20:56,349::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:20:56,349::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:20:56,349::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438856, 'last-update-local-ts': 1384438856.3490469, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:56,349::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:20:56,350::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438849, 'last-update-local-ts': 1384438856.3490469, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:20:56,350::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:21:06,444::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:21:06,692::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:21:06,767::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:21:06,827::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:21:06,827::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:21:06,828::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:21:06,828::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:06,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:21:06,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:21:06,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 16:21:06,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:06,830::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438866|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438866 (Thu Nov 14 16:21:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:21:06,831::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:21:06,838::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:21:06,838::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:21:06,838::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438866, 'last-update-local-ts': 1384438866.837954, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:06,838::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:21:06,838::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438860, 'last-update-local-ts': 1384438866.837954, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:06,839::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:21:16,966::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:21:17,228::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:21:17,304::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:21:17,365::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:21:17,365::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:21:17,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:21:17,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:17,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:21:17,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:21:17,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 16:21:17,373::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:17,374::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438877|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438877 (Thu Nov 14 16:21:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:21:17,374::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:21:17,382::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:21:17,382::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:21:17,383::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438877, 'last-update-local-ts': 1384438877.382514, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:17,383::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:21:17,383::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438870, 'last-update-local-ts': 1384438877.382514, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:17,383::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:21:27,469::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:21:27,724::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:21:27,779::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:21:27,838::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:21:27,838::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:21:27,839::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:21:27,839::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:27,839::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:21:27,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:21:27,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 16:21:27,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:27,841::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438887|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438887 (Thu Nov 14 16:21:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:21:27,841::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:21:27,850::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:21:27,850::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:21:27,850::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438887, 'last-update-local-ts': 1384438887.8501079, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:27,850::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:21:27,851::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438880, 'last-update-local-ts': 1384438887.8501079, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:27,851::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:21:37,944::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:21:38,325::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:21:38,406::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:21:38,468::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:21:38,468::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:21:38,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:21:38,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:38,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:21:38,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:21:38,470::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:21:38,470::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:38,470::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438898|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438898 (Thu Nov 14 16:21:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:21:38,471::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:21:38,479::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:21:38,480::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:21:38,480::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438898, 'last-update-local-ts': 1384438898.4796209, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:38,480::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:21:38,480::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438891, 'last-update-local-ts': 1384438898.4796209, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:38,480::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:21:48,562::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:21:48,818::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:21:48,908::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:21:48,969::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:21:48,969::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:21:48,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:21:48,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:48,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:21:48,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:21:48,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 16:21:48,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:48,978::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438908|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438908 (Thu Nov 14 16:21:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:21:48,978::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:21:48,988::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:21:48,988::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:21:48,988::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438908, 'last-update-local-ts': 1384438908.9881489, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:48,989::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:21:48,989::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438901, 'last-update-local-ts': 1384438908.9881489, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:48,989::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:21:59,075::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:21:59,330::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:21:59,397::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:21:59,473::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:21:59,473::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:21:59,473::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:21:59,474::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:59,474::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:21:59,474::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:21:59,475::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 16:21:59,475::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:21:59,476::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438919|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438919 (Thu Nov 14 16:21:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:21:59,476::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:21:59,485::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:21:59,485::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:21:59,485::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438919, 'last-update-local-ts': 1384438919.485323, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:59,486::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:21:59,486::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438912, 'last-update-local-ts': 1384438919.485323, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:21:59,486::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:22:09,581::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:22:09,849::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:22:09,924::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:22:09,985::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:22:09,985::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:22:09,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:22:09,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:09,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:22:09,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:22:09,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 16:22:09,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:09,987::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438929|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438929 (Thu Nov 14 16:22:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:22:09,988::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:22:09,996::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:22:09,997::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:22:09,997::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438929, 'last-update-local-ts': 1384438929.9965501, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:09,997::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:22:09,997::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438922, 'last-update-local-ts': 1384438929.9965501, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:09,997::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:22:20,079::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:22:20,334::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:22:20,426::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:22:20,486::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:22:20,486::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:22:20,486::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:22:20,487::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:20,487::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:22:20,487::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:22:20,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 16:22:20,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:20,488::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438940|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438940 (Thu Nov 14 16:22:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:22:20,488::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:22:20,498::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:22:20,498::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:22:20,498::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438940, 'last-update-local-ts': 1384438940.498313, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:20,499::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:22:20,499::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438933, 'last-update-local-ts': 1384438940.498313, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:20,499::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:22:30,603::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:22:30,854::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:22:30,931::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:22:30,992::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:22:30,993::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:22:30,993::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:22:30,994::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:30,994::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:22:30,994::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:22:30,994::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 16:22:30,995::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:30,996::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438950|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438950 (Thu Nov 14 16:22:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:22:30,996::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:22:31,005::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:22:31,005::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:22:31,005::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438950, 'last-update-local-ts': 1384438951.0050161, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:31,005::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:22:31,006::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438943, 'last-update-local-ts': 1384438951.0050161, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:31,006::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:22:41,078::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:22:41,333::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:22:41,421::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:22:41,482::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:22:41,482::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:22:41,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:22:41,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:41,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:22:41,490::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:22:41,490::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 16:22:41,490::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:41,491::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438961|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438961 (Thu Nov 14 16:22:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:22:41,491::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:22:41,500::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:22:41,500::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:22:41,501::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438961, 'last-update-local-ts': 1384438961.5005109, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:41,501::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:22:41,501::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438954, 'last-update-local-ts': 1384438961.5005109, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:41,501::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:22:51,577::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:22:51,831::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:22:51,907::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:22:51,968::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:22:51,969::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:22:51,969::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:22:51,969::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:51,970::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:22:51,970::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:22:51,970::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 16:22:51,971::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:22:51,972::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438971|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438971 (Thu Nov 14 16:22:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:22:51,972::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:22:51,981::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:22:51,982::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:22:51,982::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438971, 'last-update-local-ts': 1384438971.9818599, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:51,982::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:22:51,982::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438964, 'last-update-local-ts': 1384438971.9818599, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:22:51,983::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:23:02,059::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:23:02,315::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:23:02,371::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:23:02,432::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:23:02,432::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:23:02,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:23:02,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:02,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:23:02,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:23:02,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:23:02,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:02,435::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438982|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438982 (Thu Nov 14 16:23:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:23:02,435::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:23:02,444::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:23:02,444::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:23:02,444::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438982, 'last-update-local-ts': 1384438982.443923, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:02,444::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:23:02,444::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438975, 'last-update-local-ts': 1384438982.443923, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:02,445::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:23:12,521::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:23:12,773::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:23:12,866::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:23:12,938::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:23:12,938::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:23:12,939::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:23:12,939::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:12,939::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:23:12,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:23:12,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 16:23:12,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:12,941::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384438992|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384438992 (Thu Nov 14 16:23:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:23:12,941::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:23:12,949::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:23:12,950::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:23:12,950::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384438992, 'last-update-local-ts': 1384438992.9495411, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:12,950::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:23:12,950::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438985, 'last-update-local-ts': 1384438992.9495411, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:12,950::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:23:23,033::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:23:23,298::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:23:23,395::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:23:23,495::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:23:23,496::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:23:23,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:23:23,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:23,503::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7033 >MainThread::INFO::2013-11-14 16:23:23,503::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:23:23,508::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 16:23:23,511::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:23,512::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439003|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439003 (Thu Nov 14 16:23:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7033 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:23:23,512::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:23:23,527::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:23:23,527::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:23:23,527::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439003, 'last-update-local-ts': 1384439003.52721, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:23,528::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:23:23,528::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384438995, 'last-update-local-ts': 1384439003.52721, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:23,528::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:23:33,598::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:23:33,854::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:23:33,943::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:23:34,016::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:23:34,016::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:23:34,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:23:34,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:34,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:23:34,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:23:34,018::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 16:23:34,018::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:34,018::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439014|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439014 (Thu Nov 14 16:23:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:23:34,018::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:23:34,025::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:23:34,025::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:23:34,026::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439014, 'last-update-local-ts': 1384439014.0253811, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:34,026::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:23:34,026::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439006, 'last-update-local-ts': 1384439014.0253811, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:34,026::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:23:44,094::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:23:44,360::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:23:44,435::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:23:44,521::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:23:44,521::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:23:44,522::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:23:44,522::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:44,523::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:23:44,523::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:23:44,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:23:44,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:44,530::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439024|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439024 (Thu Nov 14 16:23:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:23:44,530::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:23:44,542::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:23:44,542::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:23:44,543::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439024, 'last-update-local-ts': 1384439024.542496, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:44,543::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:23:44,543::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439016, 'last-update-local-ts': 1384439024.542496, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:44,543::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:23:54,652::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:23:54,912::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:23:55,015::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:23:55,087::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:23:55,087::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:23:55,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:23:55,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:55,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:23:55,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:23:55,089::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 16:23:55,089::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:23:55,090::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439035|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439035 (Thu Nov 14 16:23:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:23:55,090::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:23:55,106::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:23:55,107::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:23:55,107::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439035, 'last-update-local-ts': 1384439035.1065061, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:55,107::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:23:55,107::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439037, 'last-update-local-ts': 1384439035.1065061, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:23:55,107::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:24:05,182::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:24:05,434::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:24:05,512::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:24:05,572::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:24:05,572::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:24:05,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:24:05,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:05,573::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:24:05,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:24:05,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 16:24:05,574::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:05,575::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439045|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439045 (Thu Nov 14 16:24:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:24:05,575::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:24:05,589::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:24:05,589::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:24:05,589::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439045, 'last-update-local-ts': 1384439045.5890801, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:05,589::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:24:05,590::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439048, 'last-update-local-ts': 1384439045.5890801, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:05,590::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:24:15,673::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:24:15,931::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:24:16,013::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:24:16,073::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:24:16,073::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:24:16,073::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:24:16,074::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:16,074::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:24:16,074::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:24:16,074::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 16:24:16,075::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:16,076::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439056|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439056 (Thu Nov 14 16:24:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:24:16,076::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:24:16,084::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:24:16,085::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:24:16,085::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439056, 'last-update-local-ts': 1384439056.084743, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:16,085::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:24:16,085::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439058, 'last-update-local-ts': 1384439056.084743, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:16,086::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:24:26,174::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:24:26,427::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:24:26,510::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:24:26,578::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:24:26,578::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:24:26,579::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:24:26,579::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:26,579::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:24:26,580::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:24:26,580::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:24:26,580::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:26,580::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439066|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439066 (Thu Nov 14 16:24:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:24:26,581::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:24:26,589::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:24:26,589::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:24:26,590::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439066, 'last-update-local-ts': 1384439066.5894129, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:26,590::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:24:26,590::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439069, 'last-update-local-ts': 1384439066.5894129, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:26,590::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:24:36,676::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:24:36,930::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:24:37,002::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:24:37,063::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:24:37,064::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:24:37,070::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:24:37,070::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:37,071::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:24:37,071::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:24:37,071::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 16:24:37,072::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:37,072::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439077|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439077 (Thu Nov 14 16:24:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:24:37,072::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:24:37,082::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:24:37,082::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:24:37,082::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439077, 'last-update-local-ts': 1384439077.0821891, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:37,083::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:24:37,083::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439079, 'last-update-local-ts': 1384439077.0821891, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:37,083::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:24:47,156::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:24:47,410::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:24:47,469::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:24:47,528::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:24:47,528::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:24:47,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:24:47,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:47,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:24:47,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:24:47,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 16:24:47,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:47,531::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439087|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439087 (Thu Nov 14 16:24:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:24:47,531::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:24:47,539::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:24:47,539::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:24:47,540::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439087, 'last-update-local-ts': 1384439087.53953, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:47,540::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:24:47,540::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439089, 'last-update-local-ts': 1384439087.53953, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:47,540::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:24:57,632::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:24:57,880::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:24:57,971::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:24:58,042::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:24:58,043::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:24:58,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:24:58,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:58,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:24:58,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:24:58,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 16:24:58,045::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:24:58,045::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439098|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439098 (Thu Nov 14 16:24:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:24:58,046::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:24:58,054::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:24:58,054::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:24:58,054::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439098, 'last-update-local-ts': 1384439098.0541351, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:58,055::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:24:58,055::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439100, 'last-update-local-ts': 1384439098.0541351, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:24:58,055::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:25:08,140::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:25:08,426::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:25:08,550::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:25:08,618::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:25:08,619::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:25:08,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:25:08,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:08,620::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:25:08,620::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:25:08,620::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 16:25:08,621::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:08,622::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439108|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439108 (Thu Nov 14 16:25:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:25:08,622::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:25:08,631::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:25:08,631::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:25:08,631::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439108, 'last-update-local-ts': 1384439108.6311519, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:08,631::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:25:08,632::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439110, 'last-update-local-ts': 1384439108.6311519, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:08,632::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:25:18,708::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:25:18,962::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:25:19,061::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:25:19,123::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:25:19,123::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:25:19,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:25:19,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:19,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:25:19,125::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:25:19,125::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 16:25:19,125::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:19,126::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439119|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439119 (Thu Nov 14 16:25:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:25:19,126::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:25:19,134::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:25:19,135::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:25:19,135::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439119, 'last-update-local-ts': 1384439119.1346171, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:19,135::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:25:19,135::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439121, 'last-update-local-ts': 1384439119.1346171, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:19,135::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:25:29,201::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:25:29,455::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:25:29,547::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:25:29,612::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:25:29,612::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:25:29,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:25:29,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:29,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:25:29,614::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:25:29,614::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 16:25:29,614::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:29,615::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439129|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439129 (Thu Nov 14 16:25:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:25:29,615::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:25:29,625::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:25:29,625::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:25:29,625::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439129, 'last-update-local-ts': 1384439129.6250679, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:29,625::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:25:29,626::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439131, 'last-update-local-ts': 1384439129.6250679, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:29,626::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:25:39,700::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:25:39,955::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:25:40,021::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:25:40,081::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:25:40,081::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:25:40,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:25:40,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:40,082::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:25:40,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:25:40,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.01 >MainThread::INFO::2013-11-14 16:25:40,083::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:40,083::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439140|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439140 (Thu Nov 14 16:25:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.01 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:25:40,084::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:25:40,092::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:25:40,092::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:25:40,092::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439140, 'last-update-local-ts': 1384439140.0923259, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:40,093::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:25:40,093::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439142, 'last-update-local-ts': 1384439140.0923259, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:40,093::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:25:50,158::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:25:50,420::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:25:50,473::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:25:50,539::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:25:50,540::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:25:50,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:25:50,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:50,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:25:50,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:25:50,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.01 >MainThread::INFO::2013-11-14 16:25:50,541::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:25:50,542::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439150|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439150 (Thu Nov 14 16:25:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.01 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:25:50,542::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:25:50,550::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:25:50,551::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:25:50,551::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439150, 'last-update-local-ts': 1384439150.5508599, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:50,551::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:25:50,551::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439152, 'last-update-local-ts': 1384439150.5508599, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:25:50,552::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:26:00,663::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:26:00,937::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:26:01,069::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:26:01,131::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:26:01,132::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:26:01,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:26:01,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:01,133::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:26:01,133::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:26:01,133::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.005 >MainThread::INFO::2013-11-14 16:26:01,134::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:01,135::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439161|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439161 (Thu Nov 14 16:26:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.005 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:26:01,135::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:26:01,143::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:26:01,144::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:26:01,144::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439161, 'last-update-local-ts': 1384439161.1438789, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:01,144::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:26:01,144::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439163, 'last-update-local-ts': 1384439161.1438789, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:01,145::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:26:11,227::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:26:11,482::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:26:11,588::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:26:11,666::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:26:11,666::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:26:11,667::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:26:11,667::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:11,667::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:26:11,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:26:11,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 16:26:11,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:11,670::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439171|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439171 (Thu Nov 14 16:26:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:26:11,670::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:26:11,677::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:26:11,677::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:26:11,677::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439171, 'last-update-local-ts': 1384439171.6771071, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:11,677::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:26:11,678::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439173, 'last-update-local-ts': 1384439171.6771071, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:11,678::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:26:21,749::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:26:21,997::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:26:22,077::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:26:22,150::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:26:22,150::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:26:22,151::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:26:22,151::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:22,151::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:26:22,152::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:26:22,152::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 16:26:22,152::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:22,153::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439182|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439182 (Thu Nov 14 16:26:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:26:22,153::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:26:22,161::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:26:22,162::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:26:22,162::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439182, 'last-update-local-ts': 1384439182.16166, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:22,162::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:26:22,162::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439184, 'last-update-local-ts': 1384439182.16166, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:22,162::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:26:32,227::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:26:32,492::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:26:32,547::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:26:32,606::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:26:32,606::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:26:32,607::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:26:32,607::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:32,607::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:26:32,608::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:26:32,608::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 16:26:32,608::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:32,609::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439192|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439192 (Thu Nov 14 16:26:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:26:32,609::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:26:32,619::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:26:32,620::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:26:32,620::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439192, 'last-update-local-ts': 1384439192.6195221, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:32,620::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:26:32,620::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439194, 'last-update-local-ts': 1384439192.6195221, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:32,620::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:26:42,703::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:26:42,953::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:26:43,049::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:26:43,109::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:26:43,109::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:26:43,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:26:43,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:43,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:26:43,111::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:26:43,111::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 16:26:43,111::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:43,112::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439203|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439203 (Thu Nov 14 16:26:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:26:43,112::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:26:43,121::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:26:43,121::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:26:43,121::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439203, 'last-update-local-ts': 1384439203.1209879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:43,121::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:26:43,122::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439204, 'last-update-local-ts': 1384439203.1209879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:43,122::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:26:53,189::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:26:53,466::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:26:53,560::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:26:53,639::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:26:53,640::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:26:53,641::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:26:53,642::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:53,643::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7033 >MainThread::INFO::2013-11-14 16:26:53,646::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:26:53,648::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 16:26:53,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:26:53,649::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439213|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439213 (Thu Nov 14 16:26:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7033 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:26:53,649::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:26:53,662::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:26:53,662::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:26:53,663::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439213, 'last-update-local-ts': 1384439213.6622839, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:53,663::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:26:53,663::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439215, 'last-update-local-ts': 1384439213.6622839, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:26:53,663::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:27:03,757::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:27:04,009::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:27:04,113::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:27:04,175::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:27:04,175::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:27:04,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:27:04,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:04,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:27:04,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:27:04,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 16:27:04,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:04,178::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439224|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439224 (Thu Nov 14 16:27:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:27:04,178::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:27:04,185::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:27:04,185::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:27:04,185::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439224, 'last-update-local-ts': 1384439224.1852911, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:04,186::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:27:04,186::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439225, 'last-update-local-ts': 1384439224.1852911, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:04,186::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:27:14,256::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:27:14,504::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:27:14,575::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:27:14,636::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:27:14,636::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:27:14,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:27:14,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:14,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:27:14,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:27:14,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 16:27:14,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:14,638::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439234|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439234 (Thu Nov 14 16:27:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:27:14,639::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:27:14,647::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:27:14,647::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:27:14,647::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439234, 'last-update-local-ts': 1384439234.6470301, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:14,647::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:27:14,648::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439236, 'last-update-local-ts': 1384439234.6470301, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:14,648::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:27:24,735::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:27:24,987::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:27:25,077::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:27:25,136::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:27:25,136::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:27:25,137::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:27:25,137::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:25,137::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:27:25,138::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:27:25,138::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:27:25,138::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:25,139::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439245|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439245 (Thu Nov 14 16:27:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:27:25,139::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:27:25,147::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:27:25,148::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:27:25,148::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439245, 'last-update-local-ts': 1384439245.1477411, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:25,148::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:27:25,148::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439246, 'last-update-local-ts': 1384439245.1477411, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:25,149::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:27:35,248::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:27:35,497::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:27:35,575::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:27:35,637::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:27:35,637::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:27:35,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:27:35,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:35,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:27:35,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:27:35,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 16:27:35,639::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:35,640::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439255|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439255 (Thu Nov 14 16:27:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:27:35,640::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:27:35,648::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:27:35,649::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:27:35,649::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439255, 'last-update-local-ts': 1384439255.648802, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:35,649::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:27:35,649::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439257, 'last-update-local-ts': 1384439255.648802, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:35,650::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:27:45,731::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:27:45,984::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:27:46,087::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:27:46,147::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:27:46,147::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:27:46,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:27:46,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:46,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:27:46,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:27:46,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 16:27:46,156::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:46,156::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439266|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439266 (Thu Nov 14 16:27:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:27:46,156::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:27:46,164::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:27:46,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:27:46,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439266, 'last-update-local-ts': 1384439266.164264, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:46,165::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:27:46,165::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439267, 'last-update-local-ts': 1384439266.164264, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:46,165::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:27:56,240::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:27:56,492::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:27:56,566::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:27:56,627::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:27:56,627::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:27:56,628::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:27:56,628::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:56,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:27:56,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:27:56,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 16:27:56,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:27:56,630::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439276|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439276 (Thu Nov 14 16:27:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:27:56,631::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:27:56,638::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:27:56,638::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:27:56,639::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439276, 'last-update-local-ts': 1384439276.6384521, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:56,639::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:27:56,639::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439278, 'last-update-local-ts': 1384439276.6384521, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:27:56,639::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:28:06,711::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:28:06,965::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:28:07,025::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:28:07,085::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:28:07,086::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:28:07,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:28:07,086::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:07,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:28:07,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:28:07,087::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 16:28:07,088::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:07,088::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439287|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439287 (Thu Nov 14 16:28:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:28:07,088::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:28:07,095::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:28:07,095::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:28:07,096::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439287, 'last-update-local-ts': 1384439287.095432, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:07,096::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:28:07,096::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439288, 'last-update-local-ts': 1384439287.095432, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:07,096::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:28:17,184::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:28:17,432::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:28:17,524::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:28:17,584::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:28:17,584::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:28:17,591::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:28:17,591::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:17,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:28:17,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:28:17,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 16:28:17,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:17,593::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439297|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439297 (Thu Nov 14 16:28:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:28:17,593::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:28:17,602::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:28:17,602::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:28:17,602::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439297, 'last-update-local-ts': 1384439297.602006, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:17,602::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:28:17,603::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439299, 'last-update-local-ts': 1384439297.602006, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:17,603::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:28:27,688::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:28:27,940::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:28:28,035::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:28:28,095::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:28:28,095::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:28:28,096::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:28:28,096::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:28,097::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:28:28,097::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:28:28,097::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 16:28:28,097::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:28,098::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439308|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439308 (Thu Nov 14 16:28:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:28:28,099::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:28:28,105::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:28:28,106::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:28:28,106::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439308, 'last-update-local-ts': 1384439308.105866, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:28,106::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:28:28,106::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439309, 'last-update-local-ts': 1384439308.105866, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:28,107::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:28:38,182::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:28:38,468::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:28:38,561::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:28:38,622::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:28:38,622::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:28:38,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:28:38,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:38,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:28:38,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:28:38,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 16:28:38,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:38,631::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439318|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439318 (Thu Nov 14 16:28:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:28:38,631::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:28:38,641::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:28:38,641::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:28:38,641::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439318, 'last-update-local-ts': 1384439318.640913, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:38,641::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:28:38,641::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439319, 'last-update-local-ts': 1384439318.640913, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:38,642::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:28:48,714::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:28:48,983::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:28:49,061::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:28:49,122::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:28:49,122::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:28:49,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:28:49,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:49,123::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:28:49,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:28:49,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:28:49,124::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:49,125::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439329|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439329 (Thu Nov 14 16:28:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:28:49,125::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:28:49,134::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:28:49,134::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:28:49,134::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439329, 'last-update-local-ts': 1384439329.1339951, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:49,134::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:28:49,135::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439330, 'last-update-local-ts': 1384439329.1339951, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:49,135::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:28:59,210::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:28:59,467::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:28:59,529::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:28:59,590::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:28:59,590::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:28:59,591::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:28:59,591::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:59,591::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:28:59,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:28:59,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 16:28:59,592::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:28:59,593::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439339|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439339 (Thu Nov 14 16:28:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:28:59,593::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:28:59,604::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:28:59,604::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:28:59,604::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439339, 'last-update-local-ts': 1384439339.604157, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:59,604::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:28:59,605::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439340, 'last-update-local-ts': 1384439339.604157, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:28:59,605::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:29:09,693::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:29:09,941::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:29:10,032::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:29:10,092::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:29:10,093::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:29:10,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:29:10,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:10,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:29:10,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:29:10,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 16:29:10,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:10,101::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439350|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439350 (Thu Nov 14 16:29:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:29:10,101::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:29:10,110::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:29:10,111::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:29:10,111::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439350, 'last-update-local-ts': 1384439350.1107111, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:10,111::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:29:10,111::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439351, 'last-update-local-ts': 1384439350.1107111, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:10,112::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:29:20,198::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:29:20,450::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:29:20,538::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:29:20,600::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:29:20,600::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:29:20,601::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:29:20,601::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:20,601::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:29:20,602::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:29:20,602::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 16:29:20,602::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:20,603::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439360|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439360 (Thu Nov 14 16:29:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:29:20,604::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:29:20,612::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:29:20,613::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:29:20,613::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439360, 'last-update-local-ts': 1384439360.6126201, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:20,613::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:29:20,613::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439361, 'last-update-local-ts': 1384439360.6126201, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:20,613::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:29:30,690::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:29:30,941::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:29:31,029::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:29:31,094::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:29:31,094::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:29:31,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:29:31,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:31,095::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:29:31,096::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:29:31,096::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.355 >MainThread::INFO::2013-11-14 16:29:31,096::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:31,097::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439371|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439371 (Thu Nov 14 16:29:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.355 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:29:31,097::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:29:31,105::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:29:31,106::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:29:31,106::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439371, 'last-update-local-ts': 1384439371.1057119, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:31,106::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:29:31,106::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439372, 'last-update-local-ts': 1384439371.1057119, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:31,107::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:29:41,172::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:29:41,427::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:29:41,489::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:29:41,550::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:29:41,551::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:29:41,560::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:29:41,561::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:41,561::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:29:41,562::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:29:41,562::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 16:29:41,562::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:41,563::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439381|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439381 (Thu Nov 14 16:29:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:29:41,563::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:29:41,573::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:29:41,573::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:29:41,573::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439381, 'last-update-local-ts': 1384439381.5732219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:41,574::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:29:41,574::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439382, 'last-update-local-ts': 1384439381.5732219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:41,574::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:29:51,674::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:29:52,000::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:29:52,082::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:29:52,146::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:29:52,146::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:29:52,147::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:29:52,147::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:52,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:29:52,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:29:52,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 16:29:52,149::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:29:52,150::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439392|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439392 (Thu Nov 14 16:29:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:29:52,150::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:29:52,161::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:29:52,161::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:29:52,161::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439392, 'last-update-local-ts': 1384439392.16119, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:52,162::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:29:52,162::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439393, 'last-update-local-ts': 1384439392.16119, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:29:52,162::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:30:02,238::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:30:02,499::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:30:02,611::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:30:02,672::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:30:02,672::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:30:02,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:30:02,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:02,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:30:02,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:30:02,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 16:30:02,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:02,681::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439402|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439402 (Thu Nov 14 16:30:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:30:02,681::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:30:02,690::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:30:02,690::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:30:02,690::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439402, 'last-update-local-ts': 1384439402.6900189, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:02,690::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:30:02,691::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439403, 'last-update-local-ts': 1384439402.6900189, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:02,691::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:30:12,773::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:30:13,028::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:30:13,158::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:30:13,217::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:30:13,218::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:30:13,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:30:13,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:13,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:30:13,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:30:13,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 16:30:13,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:13,220::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439413|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439413 (Thu Nov 14 16:30:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:30:13,221::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:30:13,227::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:30:13,228::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:30:13,228::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439413, 'last-update-local-ts': 1384439413.227618, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:13,228::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:30:13,228::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439413, 'last-update-local-ts': 1384439413.227618, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:13,228::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:30:23,303::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:30:23,569::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:30:23,660::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:30:23,724::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:30:23,724::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:30:23,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:30:23,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:23,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7032 >MainThread::INFO::2013-11-14 16:30:23,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:30:23,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:30:23,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:23,728::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439423|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439423 (Thu Nov 14 16:30:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7032 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:30:23,728::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:30:23,745::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:30:23,745::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:30:23,746::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439423, 'last-update-local-ts': 1384439423.745342, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:23,746::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:30:23,746::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439424, 'last-update-local-ts': 1384439423.745342, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:23,746::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:30:33,823::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:30:34,092::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:30:34,185::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:30:34,246::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:30:34,246::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:30:34,247::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:30:34,247::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:34,247::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:30:34,248::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:30:34,248::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 16:30:34,248::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:34,249::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439434|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439434 (Thu Nov 14 16:30:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:30:34,250::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:30:34,265::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:30:34,266::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:30:34,266::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439434, 'last-update-local-ts': 1384439434.265625, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:34,266::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:30:34,266::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439434, 'last-update-local-ts': 1384439434.265625, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:34,267::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:30:44,334::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:30:44,592::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:30:44,646::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:30:44,706::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:30:44,706::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:30:44,707::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:30:44,707::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:44,707::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:30:44,708::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:30:44,708::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 16:30:44,708::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:44,709::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439444|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439444 (Thu Nov 14 16:30:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:30:44,709::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:30:44,717::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:30:44,717::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:30:44,718::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439444, 'last-update-local-ts': 1384439444.717485, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:44,718::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:30:44,718::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439445, 'last-update-local-ts': 1384439444.717485, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:44,718::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:30:54,787::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:30:55,041::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:30:55,130::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:30:55,212::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:30:55,212::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:30:55,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:30:55,213::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:55,213::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:30:55,213::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:30:55,214::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 16:30:55,214::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:30:55,214::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439455|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439455 (Thu Nov 14 16:30:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:30:55,214::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:30:55,223::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:30:55,223::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:30:55,223::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439455, 'last-update-local-ts': 1384439455.2230301, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:55,223::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:30:55,224::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439455, 'last-update-local-ts': 1384439455.2230301, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:30:55,224::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:31:05,330::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:31:05,588::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:31:05,684::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:31:05,745::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:31:05,745::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:31:05,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:31:05,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:05,746::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:31:05,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:31:05,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:31:05,747::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:05,748::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439465|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439465 (Thu Nov 14 16:31:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:31:05,748::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:31:05,757::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:31:05,757::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:31:05,757::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439465, 'last-update-local-ts': 1384439465.7570269, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:05,757::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:31:05,758::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439466, 'last-update-local-ts': 1384439465.7570269, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:05,758::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:31:15,829::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:31:16,076::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:31:16,163::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:31:16,223::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:31:16,224::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:31:16,224::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:31:16,224::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:16,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:31:16,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:31:16,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 16:31:16,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:16,226::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439476|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439476 (Thu Nov 14 16:31:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:31:16,226::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:31:16,235::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:31:16,235::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:31:16,235::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439476, 'last-update-local-ts': 1384439476.2349491, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:16,235::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:31:16,235::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439476, 'last-update-local-ts': 1384439476.2349491, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:16,236::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:31:26,310::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:31:26,566::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:31:26,656::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:31:26,722::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:31:26,722::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:31:26,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:31:26,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:26,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:31:26,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:31:26,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 16:31:26,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:26,725::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439486|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439486 (Thu Nov 14 16:31:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:31:26,726::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:31:26,734::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:31:26,735::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:31:26,735::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439486, 'last-update-local-ts': 1384439486.7348051, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:26,735::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:31:26,735::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439487, 'last-update-local-ts': 1384439486.7348051, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:26,736::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:31:36,811::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:31:37,060::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:31:37,115::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:31:37,174::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:31:37,175::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:31:37,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:31:37,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:37,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:31:37,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:31:37,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 16:31:37,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:37,177::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439497|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439497 (Thu Nov 14 16:31:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:31:37,177::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:31:37,186::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:31:37,186::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:31:37,186::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439497, 'last-update-local-ts': 1384439497.186049, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:37,186::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:31:37,187::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439497, 'last-update-local-ts': 1384439497.186049, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:37,187::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:31:47,262::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:31:47,516::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:31:47,606::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:31:47,666::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:31:47,666::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:31:47,667::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:31:47,667::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:47,667::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:31:47,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:31:47,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 16:31:47,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:47,669::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439507|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439507 (Thu Nov 14 16:31:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:31:47,669::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:31:47,677::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:31:47,678::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:31:47,678::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439507, 'last-update-local-ts': 1384439507.6778531, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:47,678::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:31:47,678::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439508, 'last-update-local-ts': 1384439507.6778531, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:47,679::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:31:57,759::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:31:58,011::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:31:58,104::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:31:58,168::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:31:58,169::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:31:58,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:31:58,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:58,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:31:58,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:31:58,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 16:31:58,171::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:31:58,172::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439518|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439518 (Thu Nov 14 16:31:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:31:58,172::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:31:58,181::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:31:58,181::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:31:58,181::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439518, 'last-update-local-ts': 1384439518.1812229, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:58,182::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:31:58,182::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439518, 'last-update-local-ts': 1384439518.1812229, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:31:58,182::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:32:08,267::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:32:08,540::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:32:08,618::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:32:08,693::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:32:08,694::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:32:08,695::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:32:08,695::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:08,699::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:32:08,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:32:08,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 16:32:08,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:08,701::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439528|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439528 (Thu Nov 14 16:32:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:32:08,701::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:32:08,709::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:32:08,710::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:32:08,710::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439528, 'last-update-local-ts': 1384439528.7097011, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:08,710::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:32:08,710::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439528, 'last-update-local-ts': 1384439528.7097011, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:08,711::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:32:18,783::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:32:19,036::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:32:19,102::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:32:19,161::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:32:19,162::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:32:19,168::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:32:19,168::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:19,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:32:19,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:32:19,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 16:32:19,170::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:19,170::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439539|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439539 (Thu Nov 14 16:32:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:32:19,170::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:32:19,180::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:32:19,180::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:32:19,180::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439539, 'last-update-local-ts': 1384439539.1801951, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:19,181::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:32:19,181::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439539, 'last-update-local-ts': 1384439539.1801951, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:19,181::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:32:29,268::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:32:29,516::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:32:29,607::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:32:29,668::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:32:29,668::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:32:29,669::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:32:29,669::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:29,669::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:32:29,669::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:32:29,670::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 16:32:29,670::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:29,671::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439549|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439549 (Thu Nov 14 16:32:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:32:29,671::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:32:29,679::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:32:29,680::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:32:29,680::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439549, 'last-update-local-ts': 1384439549.6797819, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:29,680::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:32:29,680::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439549, 'last-update-local-ts': 1384439549.6797819, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:29,681::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:32:39,754::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:32:40,006::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:32:40,092::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:32:40,152::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:32:40,153::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:32:40,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:32:40,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:40,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:32:40,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:32:40,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:32:40,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:40,155::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439560|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439560 (Thu Nov 14 16:32:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:32:40,155::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:32:40,163::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:32:40,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:32:40,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439560, 'last-update-local-ts': 1384439560.1635721, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:40,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:32:40,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439560, 'last-update-local-ts': 1384439560.1635721, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:40,164::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:32:50,239::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:32:50,493::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:32:50,583::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:32:50,648::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:32:50,648::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:32:50,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:32:50,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:50,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:32:50,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:32:50,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 16:32:50,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:32:50,651::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439570|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439570 (Thu Nov 14 16:32:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:32:50,651::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:32:50,668::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:32:50,668::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:32:50,669::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439570, 'last-update-local-ts': 1384439570.6684599, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:50,669::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:32:50,669::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439570, 'last-update-local-ts': 1384439570.6684599, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:32:50,669::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:33:00,739::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:33:00,985::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:33:01,046::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:33:01,108::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:33:01,109::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:33:01,109::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:33:01,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:01,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:33:01,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:33:01,110::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 16:33:01,111::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:01,111::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439581|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439581 (Thu Nov 14 16:33:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:33:01,111::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:33:01,120::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:33:01,120::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:33:01,120::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439581, 'last-update-local-ts': 1384439581.1200399, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:01,120::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:33:01,121::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439581, 'last-update-local-ts': 1384439581.1200399, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:01,121::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:33:11,211::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:33:11,463::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:33:11,567::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:33:11,681::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:33:11,681::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:33:11,708::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:33:11,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:11,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:33:11,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:33:11,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:33:11,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:11,712::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439591|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439591 (Thu Nov 14 16:33:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:33:11,712::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:33:11,721::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:33:11,722::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:33:11,722::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439591, 'last-update-local-ts': 1384439591.7215581, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:11,722::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:33:11,722::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439591, 'last-update-local-ts': 1384439591.7215581, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:11,723::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:33:21,807::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:33:22,069::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:33:22,172::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:33:22,234::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:33:22,234::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:33:22,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:33:22,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:22,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:33:22,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:33:22,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 16:33:22,236::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:22,237::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439602|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439602 (Thu Nov 14 16:33:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:33:22,237::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:33:22,246::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:33:22,246::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:33:22,247::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439602, 'last-update-local-ts': 1384439602.246352, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:22,247::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:33:22,247::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439602, 'last-update-local-ts': 1384439602.246352, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:22,247::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:33:32,322::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:33:32,572::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:33:32,654::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:33:32,715::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:33:32,715::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:33:32,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:33:32,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:32,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:33:32,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:33:32,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 16:33:32,717::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:32,718::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439612|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439612 (Thu Nov 14 16:33:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:33:32,718::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:33:32,726::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:33:32,726::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:33:32,727::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439612, 'last-update-local-ts': 1384439612.7264049, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:32,727::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:33:32,727::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439612, 'last-update-local-ts': 1384439612.7264049, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:32,727::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:33:42,808::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:33:43,062::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:33:43,151::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:33:43,211::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:33:43,211::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:33:43,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:33:43,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:43,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:33:43,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:33:43,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 16:33:43,219::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:43,220::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439623|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439623 (Thu Nov 14 16:33:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:33:43,220::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:33:43,230::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:33:43,230::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:33:43,230::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439623, 'last-update-local-ts': 1384439623.2300169, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:43,230::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:33:43,231::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439622, 'last-update-local-ts': 1384439623.2300169, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:43,231::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:33:53,311::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:33:53,564::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:33:53,618::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:33:53,678::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:33:53,678::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:33:53,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:33:53,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:53,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:33:53,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:33:53,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 16:33:53,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:33:53,681::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439633|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439633 (Thu Nov 14 16:33:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:33:53,682::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:33:53,691::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:33:53,691::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:33:53,691::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439633, 'last-update-local-ts': 1384439633.6910939, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:53,691::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:33:53,692::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439633, 'last-update-local-ts': 1384439633.6910939, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:33:53,692::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:34:03,785::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:34:04,059::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:34:04,137::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:34:04,198::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:34:04,198::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:34:04,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:34:04,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:04,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:34:04,199::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:34:04,200::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 16:34:04,200::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:04,200::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439644|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439644 (Thu Nov 14 16:34:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:34:04,201::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:34:04,207::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:34:04,208::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:34:04,208::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439644, 'last-update-local-ts': 1384439644.2075579, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:04,208::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:34:04,208::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439643, 'last-update-local-ts': 1384439644.2075579, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:04,208::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:34:14,295::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:34:14,581::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:34:14,675::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:34:14,735::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:34:14,735::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:34:14,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:34:14,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:14,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:34:14,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:34:14,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.165 >MainThread::INFO::2013-11-14 16:34:14,737::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:14,737::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439654|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439654 (Thu Nov 14 16:34:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.165 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:34:14,738::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:34:14,745::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:34:14,745::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:34:14,745::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439654, 'last-update-local-ts': 1384439654.745266, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:14,746::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:34:14,746::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439654, 'last-update-local-ts': 1384439654.745266, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:14,746::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:34:24,845::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:34:25,092::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:34:25,168::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:34:25,228::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:34:25,228::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:34:25,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:34:25,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:25,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:34:25,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:34:25,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 16:34:25,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:25,231::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439665|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439665 (Thu Nov 14 16:34:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:34:25,232::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:34:25,240::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:34:25,241::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:34:25,241::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439665, 'last-update-local-ts': 1384439665.2406001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:25,241::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:34:25,241::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439664, 'last-update-local-ts': 1384439665.2406001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:25,241::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:34:35,316::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:34:35,578::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:34:35,662::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:34:35,723::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:34:35,723::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:34:35,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:34:35,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:35,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:34:35,730::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:34:35,731::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 16:34:35,731::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:35,731::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439675|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439675 (Thu Nov 14 16:34:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:34:35,732::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:34:35,740::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:34:35,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:34:35,741::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439675, 'last-update-local-ts': 1384439675.7404261, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:35,741::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:34:35,741::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439675, 'last-update-local-ts': 1384439675.7404261, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:35,741::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:34:45,815::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:34:46,069::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:34:46,119::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:34:46,178::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:34:46,179::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:34:46,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:34:46,179::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:46,180::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:34:46,180::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:34:46,180::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 16:34:46,180::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:46,181::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439686|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439686 (Thu Nov 14 16:34:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:34:46,181::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:34:46,190::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:34:46,190::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:34:46,190::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439686, 'last-update-local-ts': 1384439686.1900649, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:46,190::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:34:46,191::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439685, 'last-update-local-ts': 1384439686.1900649, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:46,192::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:34:56,282::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:34:56,528::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:34:56,617::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:34:56,678::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:34:56,679::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:34:56,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:34:56,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:56,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:34:56,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:34:56,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.195 >MainThread::INFO::2013-11-14 16:34:56,680::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:34:56,681::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439696|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439696 (Thu Nov 14 16:34:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.195 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:34:56,681::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:34:56,690::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:34:56,690::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:34:56,690::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439696, 'last-update-local-ts': 1384439696.6899731, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:56,690::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:34:56,691::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439695, 'last-update-local-ts': 1384439696.6899731, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:34:56,691::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:35:06,779::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:35:07,031::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:35:07,121::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:35:07,181::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:35:07,182::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:35:07,188::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:35:07,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:07,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:35:07,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:35:07,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.165 >MainThread::INFO::2013-11-14 16:35:07,190::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:07,190::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439707|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439707 (Thu Nov 14 16:35:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.165 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:35:07,190::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:35:07,200::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:35:07,201::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:35:07,201::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439707, 'last-update-local-ts': 1384439707.200789, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:07,201::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:35:07,201::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439706, 'last-update-local-ts': 1384439707.200789, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:07,202::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:35:17,283::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:35:17,542::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:35:17,616::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:35:17,677::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:35:17,677::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:35:17,678::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:35:17,678::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:17,678::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:35:17,678::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:35:17,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 16:35:17,679::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:17,680::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439717|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439717 (Thu Nov 14 16:35:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:35:17,680::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:35:17,688::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:35:17,688::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:35:17,688::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439717, 'last-update-local-ts': 1384439717.6880989, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:17,688::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:35:17,689::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439716, 'last-update-local-ts': 1384439717.6880989, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:17,689::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:35:27,768::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:35:28,019::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:35:28,104::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:35:28,167::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:35:28,167::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:35:28,167::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:35:28,168::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:28,168::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:35:28,168::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:35:28,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.22 >MainThread::INFO::2013-11-14 16:35:28,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:28,169::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439728|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439728 (Thu Nov 14 16:35:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.22 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:35:28,169::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:35:28,176::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:35:28,177::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:35:28,177::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439728, 'last-update-local-ts': 1384439728.17664, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:28,177::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:35:28,177::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439727, 'last-update-local-ts': 1384439728.17664, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:28,177::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:35:38,265::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:35:38,514::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:35:38,586::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:35:38,649::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:35:38,649::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:35:38,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:35:38,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:38,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:35:38,650::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:35:38,651::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.345 >MainThread::INFO::2013-11-14 16:35:38,651::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:38,651::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439738|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439738 (Thu Nov 14 16:35:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.345 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:35:38,652::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:35:38,660::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:35:38,660::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:35:38,660::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439738, 'last-update-local-ts': 1384439738.6602299, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:38,661::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:35:38,661::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439737, 'last-update-local-ts': 1384439738.6602299, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:38,661::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:35:48,760::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:35:49,022::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:35:49,119::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:35:49,179::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:35:49,180::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:35:49,180::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:35:49,180::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:49,181::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:35:49,181::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:35:49,181::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 16:35:49,181::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:49,182::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439749|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439749 (Thu Nov 14 16:35:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:35:49,183::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:35:49,190::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:35:49,191::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:35:49,191::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439749, 'last-update-local-ts': 1384439749.1906259, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:49,191::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:35:49,191::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439748, 'last-update-local-ts': 1384439749.1906259, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:49,191::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:35:59,268::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:35:59,525::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:35:59,611::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:35:59,672::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:35:59,672::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:35:59,673::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:35:59,673::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:59,673::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:35:59,674::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:35:59,674::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.43 >MainThread::INFO::2013-11-14 16:35:59,674::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:35:59,674::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439759|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439759 (Thu Nov 14 16:35:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.43 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:35:59,675::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:35:59,683::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:35:59,683::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:35:59,683::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439759, 'last-update-local-ts': 1384439759.682961, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:59,683::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:35:59,683::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439758, 'last-update-local-ts': 1384439759.682961, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:35:59,684::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:36:09,761::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:36:10,083::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:36:10,161::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:36:10,221::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:36:10,221::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:36:10,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:36:10,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:10,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:36:10,223::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:36:10,223::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.365 >MainThread::INFO::2013-11-14 16:36:10,223::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:10,224::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439770|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439770 (Thu Nov 14 16:36:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.365 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:36:10,225::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:36:10,234::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:36:10,235::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:36:10,235::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439770, 'last-update-local-ts': 1384439770.234741, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:10,235::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:36:10,235::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439769, 'last-update-local-ts': 1384439770.234741, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:10,236::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:36:20,316::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:36:20,586::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:36:20,646::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:36:20,708::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:36:20,708::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:36:20,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:36:20,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:20,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:36:20,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:36:20,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.305 >MainThread::INFO::2013-11-14 16:36:20,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:20,711::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439780|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439780 (Thu Nov 14 16:36:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.305 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:36:20,711::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:36:20,720::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:36:20,721::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:36:20,721::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439780, 'last-update-local-ts': 1384439780.7208309, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:20,721::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:36:20,721::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439779, 'last-update-local-ts': 1384439780.7208309, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:20,722::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:36:30,802::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:36:31,051::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:36:31,140::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:36:31,200::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:36:31,201::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:36:31,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:36:31,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:31,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:36:31,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:36:31,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.26 >MainThread::INFO::2013-11-14 16:36:31,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:31,209::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439791|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439791 (Thu Nov 14 16:36:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.26 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:36:31,209::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:36:31,218::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:36:31,219::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:36:31,219::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439791, 'last-update-local-ts': 1384439791.218672, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:31,219::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:36:31,219::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439790, 'last-update-local-ts': 1384439791.218672, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:31,220::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:36:41,303::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:36:41,562::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:36:41,663::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:36:41,723::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:36:41,724::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:36:41,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:36:41,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:41,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 16:36:41,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:36:41,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 16:36:41,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:41,727::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439801|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439801 (Thu Nov 14 16:36:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:36:41,727::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:36:41,736::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:36:41,736::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:36:41,736::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439801, 'last-update-local-ts': 1384439801.7362521, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:41,737::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:36:41,737::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439800, 'last-update-local-ts': 1384439801.7362521, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:41,737::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:36:51,814::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:36:52,076::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:36:52,164::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:36:52,236::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:36:52,236::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:36:52,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:36:52,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:52,237::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:36:52,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:36:52,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:36:52,238::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:36:52,239::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439812|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439812 (Thu Nov 14 16:36:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:36:52,239::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:36:52,247::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:36:52,248::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:36:52,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439812, 'last-update-local-ts': 1384439812.2476201, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:52,248::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:36:52,248::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439811, 'last-update-local-ts': 1384439812.2476201, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:36:52,248::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:37:02,320::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:37:02,575::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:37:02,631::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:37:02,691::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:37:02,691::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:37:02,692::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:37:02,692::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:02,692::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:37:02,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:37:02,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 16:37:02,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:02,694::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439822|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439822 (Thu Nov 14 16:37:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:37:02,694::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:37:02,704::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:37:02,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:37:02,704::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439822, 'last-update-local-ts': 1384439822.7040291, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:02,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:37:02,705::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439821, 'last-update-local-ts': 1384439822.7040291, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:02,705::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:37:12,798::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:37:13,046::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:37:13,137::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:37:13,215::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:37:13,216::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:37:13,216::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:37:13,217::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:13,217::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:37:13,217::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:37:13,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 16:37:13,218::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:13,219::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439833|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439833 (Thu Nov 14 16:37:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:37:13,219::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:37:13,229::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:37:13,229::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:37:13,229::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439833, 'last-update-local-ts': 1384439833.229079, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:13,230::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:37:13,230::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439831, 'last-update-local-ts': 1384439833.229079, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:13,230::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:37:23,317::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:37:23,564::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:37:23,659::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:37:23,720::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:37:23,720::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:37:23,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:37:23,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:23,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:37:23,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:37:23,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 16:37:23,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:23,728::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439843|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439843 (Thu Nov 14 16:37:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:37:23,728::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:37:23,739::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:37:23,739::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:37:23,739::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439843, 'last-update-local-ts': 1384439843.7391219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:23,739::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:37:23,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439842, 'last-update-local-ts': 1384439843.7391219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:23,740::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:37:33,827::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:37:34,166::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:37:34,266::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:37:34,327::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:37:34,327::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:37:34,328::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:37:34,328::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:34,328::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7037 >MainThread::INFO::2013-11-14 16:37:34,328::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:37:34,329::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:37:34,329::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:34,330::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439854|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439854 (Thu Nov 14 16:37:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7037 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:37:34,330::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:37:34,337::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:37:34,337::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:37:34,338::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439854, 'last-update-local-ts': 1384439854.337487, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:34,338::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:37:34,338::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439852, 'last-update-local-ts': 1384439854.337487, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:34,338::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:37:44,411::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:37:44,685::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:37:44,761::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:37:44,821::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:37:44,821::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:37:44,822::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:37:44,822::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:44,822::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:37:44,823::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:37:44,823::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 16:37:44,823::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:44,823::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439864|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439864 (Thu Nov 14 16:37:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:37:44,824::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:37:44,832::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:37:44,832::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:37:44,832::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439864, 'last-update-local-ts': 1384439864.8321991, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:44,833::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:37:44,833::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439863, 'last-update-local-ts': 1384439864.8321991, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:44,833::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:37:54,914::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:37:55,165::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:37:55,264::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:37:55,348::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:37:55,349::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:37:55,349::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:37:55,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:55,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:37:55,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:37:55,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 16:37:55,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:37:55,351::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439875|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439875 (Thu Nov 14 16:37:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:37:55,351::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:37:55,363::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:37:55,364::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:37:55,364::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439875, 'last-update-local-ts': 1384439875.3636351, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:55,364::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:37:55,364::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439873, 'last-update-local-ts': 1384439875.3636351, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:37:55,365::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:38:05,447::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:38:05,700::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:38:05,797::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:38:05,859::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:38:05,859::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:38:05,859::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:38:05,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:05,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:38:05,860::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:38:05,861::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 16:38:05,861::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:05,862::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439885|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439885 (Thu Nov 14 16:38:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:38:05,862::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:38:05,870::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:38:05,871::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:38:05,871::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439885, 'last-update-local-ts': 1384439885.8706999, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:05,871::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:38:05,871::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439884, 'last-update-local-ts': 1384439885.8706999, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:05,872::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:38:15,945::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:38:16,205::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:38:16,298::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:38:16,370::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:38:16,370::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:38:16,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:38:16,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:16,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:38:16,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:38:16,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 16:38:16,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:16,372::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439896|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439896 (Thu Nov 14 16:38:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:38:16,373::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:38:16,379::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:38:16,380::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:38:16,380::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439896, 'last-update-local-ts': 1384439896.3795619, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:16,380::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:38:16,380::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439894, 'last-update-local-ts': 1384439896.3795619, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:16,380::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:38:26,460::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:38:26,716::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:38:26,792::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:38:26,852::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:38:26,852::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:38:26,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:38:26,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:26,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:38:26,854::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:38:26,854::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 16:38:26,854::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:26,855::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439906|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439906 (Thu Nov 14 16:38:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:38:26,856::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:38:26,864::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:38:26,864::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:38:26,865::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439906, 'last-update-local-ts': 1384439906.8643789, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:26,865::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:38:26,865::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439905, 'last-update-local-ts': 1384439906.8643789, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:26,865::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:38:36,942::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:38:37,197::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:38:37,259::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:38:37,318::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:38:37,319::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:38:37,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:38:37,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:37,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:38:37,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:38:37,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.4 >MainThread::INFO::2013-11-14 16:38:37,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:37,321::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439917|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439917 (Thu Nov 14 16:38:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.4 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:38:37,321::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:38:37,329::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:38:37,330::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:38:37,330::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439917, 'last-update-local-ts': 1384439917.3296421, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:37,330::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:38:37,330::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439915, 'last-update-local-ts': 1384439917.3296421, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:37,330::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:38:47,406::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:38:47,745::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:38:47,858::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:38:47,922::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:38:47,922::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:38:47,929::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:38:47,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:47,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:38:47,930::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:38:47,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 16:38:47,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:47,931::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439927|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439927 (Thu Nov 14 16:38:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:38:47,932::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:38:47,947::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:38:47,947::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:38:47,948::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439927, 'last-update-local-ts': 1384439927.9473109, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:47,948::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:38:47,948::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439925, 'last-update-local-ts': 1384439927.9473109, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:47,948::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:38:58,024::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:38:58,282::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:38:58,372::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:38:58,432::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:38:58,432::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:38:58,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:38:58,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:58,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:38:58,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:38:58,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 16:38:58,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:38:58,435::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439938|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439938 (Thu Nov 14 16:38:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:38:58,435::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:38:58,445::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:38:58,445::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:38:58,446::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439938, 'last-update-local-ts': 1384439938.445406, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:58,446::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:38:58,446::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439936, 'last-update-local-ts': 1384439938.445406, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:38:58,446::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:39:08,525::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:39:08,778::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:39:08,891::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:39:08,960::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:39:08,961::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:39:08,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:39:08,964::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:08,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:39:08,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:39:08,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 16:39:08,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:08,969::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439948|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439948 (Thu Nov 14 16:39:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:39:08,969::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:39:08,977::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:39:08,978::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:39:08,978::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439948, 'last-update-local-ts': 1384439948.977715, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:08,978::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:39:08,978::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439946, 'last-update-local-ts': 1384439948.977715, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:08,979::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:39:19,051::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:39:19,304::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:39:19,362::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:39:19,421::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:39:19,421::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:39:19,422::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:39:19,422::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:19,422::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:39:19,423::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:39:19,423::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 16:39:19,423::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:19,424::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439959|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439959 (Thu Nov 14 16:39:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:39:19,424::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:39:19,434::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:39:19,435::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:39:19,435::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439959, 'last-update-local-ts': 1384439959.434787, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:19,435::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:39:19,435::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439957, 'last-update-local-ts': 1384439959.434787, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:19,436::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:39:29,535::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:39:29,872::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:39:29,977::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:39:30,050::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:39:30,050::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:39:30,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:39:30,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:30,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:39:30,052::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:39:30,052::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 16:39:30,052::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:30,053::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439970|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439970 (Thu Nov 14 16:39:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:39:30,054::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:39:30,064::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:39:30,064::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:39:30,065::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439970, 'last-update-local-ts': 1384439970.064328, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:30,065::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:39:30,065::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439967, 'last-update-local-ts': 1384439970.064328, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:30,065::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:39:40,140::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:39:40,386::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:39:40,465::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:39:40,526::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:39:40,526::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:39:40,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:39:40,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:40,527::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:39:40,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:39:40,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 16:39:40,528::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:40,529::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439980|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439980 (Thu Nov 14 16:39:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:39:40,529::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:39:40,536::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:39:40,537::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:39:40,537::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439980, 'last-update-local-ts': 1384439980.5367019, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:40,537::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:39:40,537::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439978, 'last-update-local-ts': 1384439980.5367019, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:40,538::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:39:50,617::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:39:50,874::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:39:50,956::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:39:51,015::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:39:51,015::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:39:51,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:39:51,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:51,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:39:51,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:39:51,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 16:39:51,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:39:51,018::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384439991|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384439991 (Thu Nov 14 16:39:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:39:51,018::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:39:51,028::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:39:51,028::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:39:51,029::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384439991, 'last-update-local-ts': 1384439991.028343, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:51,029::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:39:51,029::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439988, 'last-update-local-ts': 1384439991.028343, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:39:51,029::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:40:01,113::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:40:01,365::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:40:01,540::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:40:01,618::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:40:01,618::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:40:01,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:40:01,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:01,619::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:40:01,620::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:40:01,620::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 16:40:01,620::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:01,621::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440001|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440001 (Thu Nov 14 16:40:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:40:01,621::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:40:01,628::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:40:01,628::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:40:01,628::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440001, 'last-update-local-ts': 1384440001.6283209, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:01,629::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:40:01,629::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384439999, 'last-update-local-ts': 1384440001.6283209, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:01,629::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:40:11,727::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:40:11,982::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:40:12,093::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:40:12,154::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:40:12,154::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:40:12,161::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:40:12,161::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:12,162::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:40:12,162::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:40:12,162::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 16:40:12,162::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:12,163::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440012|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440012 (Thu Nov 14 16:40:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:40:12,164::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:40:12,173::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:40:12,174::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:40:12,174::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440012, 'last-update-local-ts': 1384440012.173872, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:12,174::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:40:12,174::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440009, 'last-update-local-ts': 1384440012.173872, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:12,175::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:40:22,248::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:40:22,502::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:40:22,562::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:40:22,629::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:40:22,629::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:40:22,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:40:22,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:22,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:40:22,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:40:22,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 16:40:22,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:22,632::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440022|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440022 (Thu Nov 14 16:40:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:40:22,632::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:40:22,640::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:40:22,641::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:40:22,641::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440022, 'last-update-local-ts': 1384440022.6406541, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:22,641::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:40:22,641::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440020, 'last-update-local-ts': 1384440022.6406541, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:22,641::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:40:32,728::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:40:32,980::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:40:33,076::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:40:33,139::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:40:33,139::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:40:33,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:40:33,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:33,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:40:33,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:40:33,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 16:40:33,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:33,142::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440033|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440033 (Thu Nov 14 16:40:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:40:33,142::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:40:33,151::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:40:33,151::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:40:33,151::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440033, 'last-update-local-ts': 1384440033.1511559, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:33,152::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:40:33,152::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440030, 'last-update-local-ts': 1384440033.1511559, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:33,152::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:40:43,240::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:40:43,487::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:40:43,567::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:40:43,628::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:40:43,629::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:40:43,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:40:43,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:43,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:40:43,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:40:43,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 16:40:43,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:43,631::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440043|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440043 (Thu Nov 14 16:40:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:40:43,632::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:40:43,640::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:40:43,641::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:40:43,641::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440043, 'last-update-local-ts': 1384440043.6407149, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:43,641::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:40:43,641::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440041, 'last-update-local-ts': 1384440043.6407149, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:43,642::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:40:53,721::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:40:53,976::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:40:54,070::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:40:54,130::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:40:54,131::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:40:54,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:40:54,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:54,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 16:40:54,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:40:54,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 16:40:54,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:40:54,133::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440054|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440054 (Thu Nov 14 16:40:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:40:54,133::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:40:54,140::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:40:54,141::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:40:54,141::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440054, 'last-update-local-ts': 1384440054.1408579, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:54,141::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:40:54,141::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440051, 'last-update-local-ts': 1384440054.1408579, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:40:54,142::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:41:04,308::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:41:04,570::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:41:04,667::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:41:04,727::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:41:04,727::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:41:04,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:41:04,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:04,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:41:04,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:41:04,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 16:41:04,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:04,730::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440064|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440064 (Thu Nov 14 16:41:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:41:04,730::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:41:04,737::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:41:04,737::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:41:04,738::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440064, 'last-update-local-ts': 1384440064.737458, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:04,738::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:41:04,738::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440061, 'last-update-local-ts': 1384440064.737458, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:04,738::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:41:14,811::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:41:15,060::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:41:15,114::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:41:15,175::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:41:15,175::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:41:15,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:41:15,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:15,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:41:15,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:41:15,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 16:41:15,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:15,177::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440075|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440075 (Thu Nov 14 16:41:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:41:15,178::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:41:15,186::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:41:15,186::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:41:15,186::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440075, 'last-update-local-ts': 1384440075.1862681, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:15,187::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:41:15,187::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440072, 'last-update-local-ts': 1384440075.1862681, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:15,187::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:41:25,265::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:41:25,517::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:41:25,607::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:41:25,667::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:41:25,667::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:41:25,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:41:25,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:25,668::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:41:25,669::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:41:25,669::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 16:41:25,669::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:25,670::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440085|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440085 (Thu Nov 14 16:41:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:41:25,670::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:41:25,679::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:41:25,679::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:41:25,679::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440085, 'last-update-local-ts': 1384440085.6793151, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:25,680::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:41:25,680::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440082, 'last-update-local-ts': 1384440085.6793151, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:25,680::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:41:35,775::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:41:36,025::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:41:36,113::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:41:36,173::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:41:36,173::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:41:36,174::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:41:36,174::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:36,174::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:41:36,174::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:41:36,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:41:36,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:36,176::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440096|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440096 (Thu Nov 14 16:41:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:41:36,177::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:41:36,185::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:41:36,186::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:41:36,186::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440096, 'last-update-local-ts': 1384440096.1857071, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:36,186::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:41:36,186::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440093, 'last-update-local-ts': 1384440096.1857071, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:36,187::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:41:46,252::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:41:46,517::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:41:46,608::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:41:46,667::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:41:46,668::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:41:46,674::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:41:46,674::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:46,675::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:41:46,675::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:41:46,675::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 16:41:46,676::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:46,676::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440106|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440106 (Thu Nov 14 16:41:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:41:46,676::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:41:46,685::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:41:46,685::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:41:46,685::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440106, 'last-update-local-ts': 1384440106.684907, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:46,685::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:41:46,685::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440103, 'last-update-local-ts': 1384440106.684907, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:46,686::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:41:56,758::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:41:57,011::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:41:57,069::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:41:57,127::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:41:57,128::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:41:57,128::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:41:57,128::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:57,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:41:57,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:41:57,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 16:41:57,129::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:41:57,131::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440117|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440117 (Thu Nov 14 16:41:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:41:57,131::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:41:57,139::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:41:57,140::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:41:57,140::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440117, 'last-update-local-ts': 1384440117.1395741, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:57,140::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:41:57,140::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440114, 'last-update-local-ts': 1384440117.1395741, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:41:57,140::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:42:07,240::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:42:07,501::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:42:07,607::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:42:07,682::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:42:07,682::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:42:07,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:42:07,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:07,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:42:07,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:42:07,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:42:07,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:07,685::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440127|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440127 (Thu Nov 14 16:42:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:42:07,685::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:42:07,703::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:42:07,703::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:42:07,703::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440127, 'last-update-local-ts': 1384440127.70296, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:07,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:42:07,704::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440124, 'last-update-local-ts': 1384440127.70296, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:07,704::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:42:17,792::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:42:18,048::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:42:18,147::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:42:18,208::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:42:18,209::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:42:18,215::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:42:18,215::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:18,216::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:42:18,216::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:42:18,216::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 16:42:18,217::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:18,217::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440138|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440138 (Thu Nov 14 16:42:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:42:18,217::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:42:18,226::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:42:18,226::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:42:18,226::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440138, 'last-update-local-ts': 1384440138.226052, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:18,226::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:42:18,227::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440135, 'last-update-local-ts': 1384440138.226052, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:18,227::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:42:28,311::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:42:28,565::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:42:28,659::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:42:28,719::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:42:28,720::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:42:28,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:42:28,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:28,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:42:28,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:42:28,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 16:42:28,721::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:28,723::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440148|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440148 (Thu Nov 14 16:42:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:42:28,723::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:42:28,730::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:42:28,730::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:42:28,730::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440148, 'last-update-local-ts': 1384440148.7299211, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:28,730::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:42:28,730::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440145, 'last-update-local-ts': 1384440148.7299211, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:28,731::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:42:38,806::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:42:39,129::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:42:39,210::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:42:39,283::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:42:39,283::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:42:39,283::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:42:39,284::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:39,284::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:42:39,284::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:42:39,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.15 >MainThread::INFO::2013-11-14 16:42:39,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:39,285::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440159|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440159 (Thu Nov 14 16:42:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.15 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:42:39,285::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:42:39,293::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:42:39,293::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:42:39,293::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440159, 'last-update-local-ts': 1384440159.2932739, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:39,294::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:42:39,294::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440156, 'last-update-local-ts': 1384440159.2932739, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:39,294::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:42:49,366::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:42:49,620::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:42:49,705::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:42:49,766::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:42:49,766::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:42:49,767::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:42:49,767::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:49,767::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:42:49,767::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:42:49,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 16:42:49,768::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:42:49,769::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440169|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440169 (Thu Nov 14 16:42:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:42:49,769::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:42:49,777::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:42:49,777::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:42:49,777::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440169, 'last-update-local-ts': 1384440169.777189, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:49,778::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:42:49,778::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440166, 'last-update-local-ts': 1384440169.777189, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:42:49,778::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:42:59,850::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:43:00,110::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:43:00,166::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:43:00,226::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:43:00,226::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:43:00,227::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:43:00,227::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:00,227::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:43:00,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:43:00,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 16:43:00,228::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:00,228::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440180|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440180 (Thu Nov 14 16:43:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:43:00,229::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:43:00,237::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:43:00,238::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:43:00,238::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440180, 'last-update-local-ts': 1384440180.237582, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:00,238::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:43:00,238::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440177, 'last-update-local-ts': 1384440180.237582, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:00,238::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:43:10,307::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:43:10,559::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:43:10,645::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:43:10,718::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:43:10,718::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:43:10,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:43:10,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:10,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:43:10,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:43:10,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.09 >MainThread::INFO::2013-11-14 16:43:10,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:10,721::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440190|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440190 (Thu Nov 14 16:43:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.09 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:43:10,721::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:43:10,729::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:43:10,730::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:43:10,730::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440190, 'last-update-local-ts': 1384440190.729594, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:10,730::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:43:10,730::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440187, 'last-update-local-ts': 1384440190.729594, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:10,730::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:43:20,813::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:43:21,066::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:43:21,171::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:43:21,231::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:43:21,231::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:43:21,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:43:21,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:21,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:43:21,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:43:21,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 16:43:21,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:21,234::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440201|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440201 (Thu Nov 14 16:43:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:43:21,234::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:43:21,241::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:43:21,242::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:43:21,242::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440201, 'last-update-local-ts': 1384440201.2416351, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:21,242::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:43:21,242::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440197, 'last-update-local-ts': 1384440201.2416351, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:21,242::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:43:31,320::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:43:31,569::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:43:31,664::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:43:31,724::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:43:31,724::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:43:31,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:43:31,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:31,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:43:31,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:43:31,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 16:43:31,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:31,727::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440211|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440211 (Thu Nov 14 16:43:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:43:31,727::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:43:31,736::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:43:31,736::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:43:31,736::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440211, 'last-update-local-ts': 1384440211.736043, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:31,736::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:43:31,737::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440208, 'last-update-local-ts': 1384440211.736043, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:31,737::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:43:41,810::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:43:42,118::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:43:42,209::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:43:42,278::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:43:42,278::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:43:42,284::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:43:42,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:42,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:43:42,285::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:43:42,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 16:43:42,286::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:42,287::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440222|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440222 (Thu Nov 14 16:43:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:43:42,287::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:43:42,297::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:43:42,297::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:43:42,297::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440222, 'last-update-local-ts': 1384440222.297204, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:42,298::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:43:42,298::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440218, 'last-update-local-ts': 1384440222.297204, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:42,298::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:43:52,398::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:43:52,677::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:43:52,731::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:43:52,795::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:43:52,795::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:43:52,795::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:43:52,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:52,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:43:52,796::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:43:52,797::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 16:43:52,797::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:43:52,798::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440232|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440232 (Thu Nov 14 16:43:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:43:52,798::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:43:52,806::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:43:52,807::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:43:52,807::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440232, 'last-update-local-ts': 1384440232.806829, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:52,807::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:43:52,807::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440229, 'last-update-local-ts': 1384440232.806829, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:43:52,808::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:44:02,871::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:44:03,188::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:44:03,261::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:44:03,321::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:44:03,321::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:44:03,322::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:44:03,322::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:03,322::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:44:03,323::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:44:03,323::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 16:44:03,323::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:03,323::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440243|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440243 (Thu Nov 14 16:44:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:44:03,324::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:44:03,330::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:44:03,331::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:44:03,331::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440243, 'last-update-local-ts': 1384440243.3306761, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:03,331::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:44:03,331::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440239, 'last-update-local-ts': 1384440243.3306761, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:03,331::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:44:13,410::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:44:13,660::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:44:13,751::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:44:13,813::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:44:13,813::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:44:13,820::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:44:13,820::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:13,820::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:44:13,821::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:44:13,821::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 16:44:13,821::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:13,822::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440253|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440253 (Thu Nov 14 16:44:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:44:13,822::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:44:13,832::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:44:13,833::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:44:13,833::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440253, 'last-update-local-ts': 1384440253.8325591, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:13,833::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:44:13,833::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440250, 'last-update-local-ts': 1384440253.8325591, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:13,833::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:44:23,922::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:44:24,179::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:44:24,300::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:44:24,385::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:44:24,386::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:44:24,387::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:44:24,387::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:24,387::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:44:24,388::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:44:24,388::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 16:44:24,388::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:24,389::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440264|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440264 (Thu Nov 14 16:44:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:44:24,390::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:44:24,397::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:44:24,397::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:44:24,398::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440264, 'last-update-local-ts': 1384440264.3972499, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:24,398::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:44:24,398::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440260, 'last-update-local-ts': 1384440264.3972499, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:24,398::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:44:34,476::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:44:34,733::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:44:34,827::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:44:34,888::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:44:34,888::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:44:34,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:44:34,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:34,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7042 >MainThread::INFO::2013-11-14 16:44:34,896::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:44:34,896::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 16:44:34,896::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:34,897::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440274|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440274 (Thu Nov 14 16:44:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7042 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:44:34,897::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:44:34,904::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:44:34,904::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:44:34,904::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440274, 'last-update-local-ts': 1384440274.904078, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:34,904::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:44:34,905::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440271, 'last-update-local-ts': 1384440274.904078, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:34,905::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:44:44,978::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:44:45,231::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:44:45,291::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:44:45,350::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:44:45,350::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:44:45,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:44:45,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:45,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:44:45,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:44:45,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 16:44:45,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:45,353::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440285|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440285 (Thu Nov 14 16:44:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:44:45,353::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:44:45,362::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:44:45,362::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:44:45,363::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440285, 'last-update-local-ts': 1384440285.3623371, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:45,363::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:44:45,363::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440281, 'last-update-local-ts': 1384440285.3623371, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:45,363::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:44:55,457::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:44:55,706::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:44:55,803::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:44:55,903::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:44:55,904::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:44:55,904::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:44:55,905::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:55,905::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:44:55,905::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:44:55,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 16:44:55,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:44:55,906::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440295|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440295 (Thu Nov 14 16:44:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:44:55,907::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:44:55,915::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:44:55,915::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:44:55,915::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440295, 'last-update-local-ts': 1384440295.9149139, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:55,915::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:44:55,916::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440291, 'last-update-local-ts': 1384440295.9149139, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:44:55,916::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:45:05,989::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:45:06,244::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:45:06,348::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:45:06,408::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:45:06,408::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:45:06,415::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:45:06,415::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:06,415::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:45:06,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:45:06,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 16:45:06,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:06,417::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440306|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440306 (Thu Nov 14 16:45:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:45:06,417::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:45:06,427::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:45:06,427::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:45:06,427::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440306, 'last-update-local-ts': 1384440306.4269249, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:06,427::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:45:06,427::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440302, 'last-update-local-ts': 1384440306.4269249, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:06,428::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:45:16,515::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:45:16,768::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:45:16,870::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:45:16,930::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:45:16,930::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:45:16,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:45:16,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:16,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:45:16,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:45:16,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 16:45:16,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:16,933::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440316|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440316 (Thu Nov 14 16:45:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:45:16,934::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:45:16,942::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:45:16,942::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:45:16,943::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440316, 'last-update-local-ts': 1384440316.9424939, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:16,943::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:45:16,943::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440312, 'last-update-local-ts': 1384440316.9424939, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:16,943::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:45:27,011::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:45:27,273::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:45:27,359::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:45:27,425::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:45:27,426::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:45:27,426::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:45:27,426::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:27,427::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:45:27,427::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:45:27,427::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 16:45:27,427::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:27,428::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440327|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440327 (Thu Nov 14 16:45:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:45:27,428::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:45:27,436::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:45:27,437::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:45:27,437::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440327, 'last-update-local-ts': 1384440327.436851, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:27,437::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:45:27,437::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440323, 'last-update-local-ts': 1384440327.436851, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:27,438::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:45:37,509::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:45:37,761::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:45:37,854::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:45:37,912::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:45:37,913::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:45:37,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:45:37,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:37,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 16:45:37,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:45:37,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 16:45:37,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:37,916::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440337|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440337 (Thu Nov 14 16:45:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:45:37,916::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:45:37,925::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:45:37,925::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:45:37,926::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440337, 'last-update-local-ts': 1384440337.925328, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:37,926::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:45:37,926::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440333, 'last-update-local-ts': 1384440337.925328, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:37,926::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:45:48,002::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:45:48,260::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:45:48,315::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:45:48,375::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:45:48,376::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:45:48,376::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:45:48,376::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:48,377::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 16:45:48,377::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:45:48,377::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 16:45:48,377::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:48,378::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440348|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440348 (Thu Nov 14 16:45:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:45:48,378::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:45:48,387::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:45:48,387::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:45:48,387::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440348, 'last-update-local-ts': 1384440348.38692, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:48,387::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:45:48,387::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440344, 'last-update-local-ts': 1384440348.38692, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:48,388::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:45:58,464::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:45:58,724::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:45:58,812::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:45:58,872::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:45:58,873::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:45:58,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:45:58,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:58,874::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:45:58,874::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:45:58,874::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.535 >MainThread::INFO::2013-11-14 16:45:58,874::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:45:58,875::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440358|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440358 (Thu Nov 14 16:45:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.535 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:45:58,875::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:45:58,883::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:45:58,884::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:45:58,884::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440358, 'last-update-local-ts': 1384440358.883734, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:58,884::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:45:58,884::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440354, 'last-update-local-ts': 1384440358.883734, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:45:58,885::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:46:08,961::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:46:09,264::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:46:09,371::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:46:09,432::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:46:09,432::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:46:09,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:46:09,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:09,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:46:09,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:46:09,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.45 >MainThread::INFO::2013-11-14 16:46:09,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:09,435::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440369|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440369 (Thu Nov 14 16:46:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.45 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:46:09,435::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:46:09,444::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:46:09,444::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:46:09,444::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440369, 'last-update-local-ts': 1384440369.444011, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:09,444::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:46:09,445::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440365, 'last-update-local-ts': 1384440369.444011, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:09,445::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:46:19,523::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:46:19,769::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:46:19,859::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:46:19,919::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:46:19,920::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:46:19,920::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:46:19,920::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:19,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:46:19,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:46:19,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.38 >MainThread::INFO::2013-11-14 16:46:19,921::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:19,922::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440379|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440379 (Thu Nov 14 16:46:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.38 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:46:19,922::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:46:19,929::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:46:19,930::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:46:19,930::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440379, 'last-update-local-ts': 1384440379.929745, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:19,930::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:46:19,930::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440375, 'last-update-local-ts': 1384440379.929745, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:19,931::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:46:30,004::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:46:30,271::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:46:30,361::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:46:30,426::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:46:30,426::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:46:30,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:46:30,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:30,433::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:46:30,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:46:30,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 16:46:30,434::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:30,435::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440390|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440390 (Thu Nov 14 16:46:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.32 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:46:30,435::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:46:30,445::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:46:30,445::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:46:30,445::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440390, 'last-update-local-ts': 1384440390.4451849, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:30,446::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:46:30,446::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440386, 'last-update-local-ts': 1384440390.4451849, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:30,446::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:46:40,522::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:46:40,774::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:46:40,836::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:46:40,897::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:46:40,897::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:46:40,897::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:46:40,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:40,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:46:40,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:46:40,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.275 >MainThread::INFO::2013-11-14 16:46:40,899::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:40,899::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440400|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440400 (Thu Nov 14 16:46:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.275 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:46:40,899::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:46:40,908::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:46:40,908::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:46:40,908::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440400, 'last-update-local-ts': 1384440400.907932, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:40,908::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:46:40,908::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440396, 'last-update-local-ts': 1384440400.907932, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:40,909::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:46:50,987::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:46:51,242::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:46:51,329::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:46:51,389::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:46:51,389::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:46:51,390::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:46:51,390::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:51,390::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:46:51,391::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:46:51,391::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 16:46:51,391::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:46:51,391::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440411|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440411 (Thu Nov 14 16:46:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:46:51,392::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:46:51,400::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:46:51,401::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:46:51,401::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440411, 'last-update-local-ts': 1384440411.400646, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:51,401::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:46:51,401::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440406, 'last-update-local-ts': 1384440411.400646, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:46:51,401::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:47:01,476::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:47:01,729::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:47:01,829::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:47:01,896::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:47:01,896::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:47:01,897::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:47:01,897::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:01,897::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:47:01,897::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:47:01,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.165 >MainThread::INFO::2013-11-14 16:47:01,898::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:01,899::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440421|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440421 (Thu Nov 14 16:47:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.165 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:47:01,899::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:47:01,908::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:47:01,909::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:47:01,909::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440421, 'last-update-local-ts': 1384440421.9088249, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:01,909::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:47:01,909::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440417, 'last-update-local-ts': 1384440421.9088249, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:01,910::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:47:11,991::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:47:12,246::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:47:12,330::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:47:12,391::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:47:12,391::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:47:12,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:47:12,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:12,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:47:12,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:47:12,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.62 >MainThread::INFO::2013-11-14 16:47:12,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:12,394::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440432|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440432 (Thu Nov 14 16:47:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.62 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:47:12,394::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:47:12,402::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:47:12,403::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:47:12,403::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440432, 'last-update-local-ts': 1384440432.4027431, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:12,403::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:47:12,403::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440427, 'last-update-local-ts': 1384440432.4027431, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:12,404::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:47:22,476::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:47:22,814::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:47:22,921::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:47:22,982::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:47:22,982::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:47:22,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:47:22,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:22,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:47:22,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:47:22,990::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.525 >MainThread::INFO::2013-11-14 16:47:22,990::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:22,991::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440442|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440442 (Thu Nov 14 16:47:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.525 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:47:22,991::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:47:23,001::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:47:23,001::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:47:23,002::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440442, 'last-update-local-ts': 1384440443.0014009, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:23,002::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:47:23,002::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440438, 'last-update-local-ts': 1384440443.0014009, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:23,002::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:47:33,098::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:47:33,375::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:47:33,425::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:47:33,485::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:47:33,485::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:47:33,486::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:47:33,486::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:33,486::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:47:33,487::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:47:33,487::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.445 >MainThread::INFO::2013-11-14 16:47:33,487::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:33,487::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440453|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440453 (Thu Nov 14 16:47:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.445 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:47:33,488::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:47:33,496::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:47:33,497::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:47:33,497::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440453, 'last-update-local-ts': 1384440453.49668, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:33,497::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:47:33,497::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440448, 'last-update-local-ts': 1384440453.49668, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:33,498::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:47:43,563::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:47:43,809::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:47:43,874::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:47:43,939::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:47:43,940::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:47:43,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:47:43,940::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:43,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:47:43,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:47:43,941::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.375 >MainThread::INFO::2013-11-14 16:47:43,942::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:43,942::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440463|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440463 (Thu Nov 14 16:47:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.375 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:47:43,942::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:47:43,951::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:47:43,952::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:47:43,952::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440463, 'last-update-local-ts': 1384440463.951628, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:43,952::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:47:43,952::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440459, 'last-update-local-ts': 1384440463.951628, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:43,952::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:47:54,030::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:47:54,277::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:47:54,362::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:47:54,422::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:47:54,422::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:47:54,423::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:47:54,423::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:54,423::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:47:54,424::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:47:54,424::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.315 >MainThread::INFO::2013-11-14 16:47:54,424::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:47:54,425::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440474|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440474 (Thu Nov 14 16:47:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.315 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:47:54,425::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:47:54,431::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:47:54,432::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:47:54,432::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440474, 'last-update-local-ts': 1384440474.4318371, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:54,432::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:47:54,432::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440469, 'last-update-local-ts': 1384440474.4318371, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:47:54,433::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:48:04,523::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:48:04,791::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:48:04,894::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:48:04,960::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:48:04,960::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:48:04,961::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:48:04,961::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:04,961::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:48:04,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:48:04,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.27 >MainThread::INFO::2013-11-14 16:48:04,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:04,963::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440484|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440484 (Thu Nov 14 16:48:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.27 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:48:04,964::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:48:04,970::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:48:04,971::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:48:04,971::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440484, 'last-update-local-ts': 1384440484.970823, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:04,971::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:48:04,971::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440480, 'last-update-local-ts': 1384440484.970823, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:04,972::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:48:15,052::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:48:15,302::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:48:15,379::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:48:15,441::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:48:15,441::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:48:15,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:48:15,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:15,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:48:15,442::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:48:15,443::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 16:48:15,443::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:15,443::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440495|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440495 (Thu Nov 14 16:48:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:48:15,444::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:48:15,452::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:48:15,452::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:48:15,452::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440495, 'last-update-local-ts': 1384440495.4519069, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:15,452::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:48:15,452::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440490, 'last-update-local-ts': 1384440495.4519069, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:15,453::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:48:25,536::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:48:25,788::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:48:25,866::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:48:25,926::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:48:25,927::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:48:25,933::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:48:25,933::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:25,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:48:25,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:48:25,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 16:48:25,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:25,935::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440505|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440505 (Thu Nov 14 16:48:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:48:25,935::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:48:25,945::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:48:25,945::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:48:25,946::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440505, 'last-update-local-ts': 1384440505.9453709, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:25,946::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:48:25,946::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440501, 'last-update-local-ts': 1384440505.9453709, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:25,946::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:48:36,019::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:48:36,279::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:48:36,332::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:48:36,391::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:48:36,391::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:48:36,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:48:36,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:36,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:48:36,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:48:36,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 16:48:36,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:36,394::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440516|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440516 (Thu Nov 14 16:48:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:48:36,395::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:48:36,412::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:48:36,413::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:48:36,413::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440516, 'last-update-local-ts': 1384440516.4124539, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:36,413::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:48:36,413::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440511, 'last-update-local-ts': 1384440516.4124539, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:36,414::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:48:46,514::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:48:46,772::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:48:46,851::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:48:46,913::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:48:46,913::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:48:46,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:48:46,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:46,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:48:46,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:48:46,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 16:48:46,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:46,916::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440526|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440526 (Thu Nov 14 16:48:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:48:46,916::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:48:46,924::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:48:46,925::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:48:46,925::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440526, 'last-update-local-ts': 1384440526.92484, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:46,925::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:48:46,925::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440522, 'last-update-local-ts': 1384440526.92484, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:46,926::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:48:57,011::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:48:57,266::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:48:57,349::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:48:57,409::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:48:57,409::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:48:57,410::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:48:57,410::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:57,410::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:48:57,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:48:57,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 16:48:57,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:48:57,411::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440537|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440537 (Thu Nov 14 16:48:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:48:57,412::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:48:57,421::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:48:57,421::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:48:57,421::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440537, 'last-update-local-ts': 1384440537.4211409, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:57,421::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:48:57,422::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440532, 'last-update-local-ts': 1384440537.4211409, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:48:57,422::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:49:07,509::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:49:07,760::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:49:07,841::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:49:07,901::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:49:07,902::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:49:07,902::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:49:07,902::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:07,903::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:49:07,903::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:49:07,903::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 16:49:07,904::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:07,905::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440547|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440547 (Thu Nov 14 16:49:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:49:07,905::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:49:07,913::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:49:07,914::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:49:07,914::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440547, 'last-update-local-ts': 1384440547.913888, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:07,914::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:49:07,914::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440543, 'last-update-local-ts': 1384440547.913888, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:07,915::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:49:17,979::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:49:18,241::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:49:18,358::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:49:18,481::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:49:18,482::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:49:18,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:49:18,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:18,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:49:18,494::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:49:18,494::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 16:49:18,494::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:18,495::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440558|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440558 (Thu Nov 14 16:49:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:49:18,495::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:49:18,507::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:49:18,507::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:49:18,507::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440558, 'last-update-local-ts': 1384440558.5071571, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:18,508::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:49:18,508::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440553, 'last-update-local-ts': 1384440558.5071571, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:18,508::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:49:28,608::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:49:28,878::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:49:28,937::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:49:29,003::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:49:29,003::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:49:29,004::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:49:29,004::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:29,005::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:49:29,005::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:49:29,005::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 16:49:29,005::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:29,007::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440569|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440569 (Thu Nov 14 16:49:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:49:29,007::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:49:29,017::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:49:29,017::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:49:29,017::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440569, 'last-update-local-ts': 1384440569.0172019, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:29,018::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:49:29,018::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440563, 'last-update-local-ts': 1384440569.0172019, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:29,018::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:49:39,093::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:49:39,360::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:49:39,428::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:49:39,490::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:49:39,490::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:49:39,490::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:49:39,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:39,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:49:39,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:49:39,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 16:49:39,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:39,492::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440579|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440579 (Thu Nov 14 16:49:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:49:39,492::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:49:39,499::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:49:39,499::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:49:39,500::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440579, 'last-update-local-ts': 1384440579.4993961, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:39,500::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:49:39,500::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440574, 'last-update-local-ts': 1384440579.4993961, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:39,500::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:49:49,598::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:49:49,848::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:49:49,929::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:49:49,996::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:49:49,996::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:49:49,997::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:49:49,997::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:49,997::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:49:49,998::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:49:49,998::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 16:49:49,998::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:49:49,998::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440589|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440589 (Thu Nov 14 16:49:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:49:49,999::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:49:50,005::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:49:50,006::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:49:50,006::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440589, 'last-update-local-ts': 1384440590.0056889, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:50,006::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:49:50,006::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440584, 'last-update-local-ts': 1384440590.0056889, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:49:50,007::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:50:00,087::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:50:00,342::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:50:00,428::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:50:00,490::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:50:00,490::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:50:00,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:50:00,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:00,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:50:00,498::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:50:00,498::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 16:50:00,498::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:00,499::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440600|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440600 (Thu Nov 14 16:50:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:50:00,499::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:50:00,508::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:50:00,508::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:50:00,509::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440600, 'last-update-local-ts': 1384440600.508348, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:00,509::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:50:00,509::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440595, 'last-update-local-ts': 1384440600.508348, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:00,509::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:50:10,602::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:50:10,857::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:50:10,953::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:50:11,014::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:50:11,014::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:50:11,020::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:50:11,021::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:11,021::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:50:11,021::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:50:11,021::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 16:50:11,022::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:11,023::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440611|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440611 (Thu Nov 14 16:50:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:50:11,023::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:50:11,032::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:50:11,032::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:50:11,033::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440611, 'last-update-local-ts': 1384440611.0324509, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:11,033::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:50:11,033::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440605, 'last-update-local-ts': 1384440611.0324509, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:11,033::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:50:21,104::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:50:21,352::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:50:21,408::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:50:21,466::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:50:21,466::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:50:21,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:50:21,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:21,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:50:21,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:50:21,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 16:50:21,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:21,470::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440621|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440621 (Thu Nov 14 16:50:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:50:21,470::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:50:21,479::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:50:21,480::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:50:21,480::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440621, 'last-update-local-ts': 1384440621.479578, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:21,480::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:50:21,480::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440616, 'last-update-local-ts': 1384440621.479578, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:21,480::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:50:31,563::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:50:31,813::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:50:31,902::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:50:31,980::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:50:31,980::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:50:31,983::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:50:31,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:31,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:50:31,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:50:31,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 16:50:31,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:31,988::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440631|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440631 (Thu Nov 14 16:50:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:50:31,988::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:50:32,006::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:50:32,006::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:50:32,007::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440631, 'last-update-local-ts': 1384440632.006326, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:32,007::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:50:32,007::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440626, 'last-update-local-ts': 1384440632.006326, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:32,007::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:50:42,096::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:50:42,349::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:50:42,436::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:50:42,496::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:50:42,497::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:50:42,503::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:50:42,503::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:42,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:50:42,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:50:42,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 16:50:42,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:42,505::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440642|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440642 (Thu Nov 14 16:50:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:50:42,505::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:50:42,515::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:50:42,516::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:50:42,516::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440642, 'last-update-local-ts': 1384440642.5155549, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:42,516::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:50:42,516::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440637, 'last-update-local-ts': 1384440642.5155549, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:42,516::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:50:52,602::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:50:52,853::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:50:52,963::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:50:53,026::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:50:53,026::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:50:53,026::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:50:53,027::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:53,027::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:50:53,027::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:50:53,028::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 16:50:53,028::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:50:53,029::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440653|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440653 (Thu Nov 14 16:50:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:50:53,029::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:50:53,038::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:50:53,038::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:50:53,038::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440653, 'last-update-local-ts': 1384440653.0379651, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:53,038::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:50:53,039::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440647, 'last-update-local-ts': 1384440653.0379651, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:50:53,039::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:51:03,123::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:51:03,386::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:51:03,480::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:51:03,546::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:51:03,546::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:51:03,553::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:51:03,553::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:03,553::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:51:03,554::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:51:03,554::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 16:51:03,554::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:03,555::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440663|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440663 (Thu Nov 14 16:51:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:51:03,555::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:51:03,564::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:51:03,564::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:51:03,564::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440663, 'last-update-local-ts': 1384440663.5639889, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:03,564::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:51:03,565::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440657, 'last-update-local-ts': 1384440663.5639889, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:03,565::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:51:13,637::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:51:13,890::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:51:13,954::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:51:14,013::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:51:14,014::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:51:14,014::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:51:14,014::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:14,015::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:51:14,015::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:51:14,015::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 16:51:14,015::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:14,017::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440674|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440674 (Thu Nov 14 16:51:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:51:14,017::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:51:14,026::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:51:14,026::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:51:14,026::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440674, 'last-update-local-ts': 1384440674.025862, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:14,026::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:51:14,027::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440668, 'last-update-local-ts': 1384440674.025862, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:14,027::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:51:24,124::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:51:24,374::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:51:24,456::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:51:24,519::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:51:24,519::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:51:24,520::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:51:24,520::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:24,521::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:51:24,521::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:51:24,521::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 16:51:24,521::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:24,522::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440684|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440684 (Thu Nov 14 16:51:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:51:24,522::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:51:24,549::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:51:24,553::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:51:24,555::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440684, 'last-update-local-ts': 1384440684.548346, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:24,557::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:51:24,560::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440678, 'last-update-local-ts': 1384440684.548346, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:24,564::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:51:34,680::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:51:34,942::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:51:35,033::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:51:35,094::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:51:35,094::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:51:35,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:51:35,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:35,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:51:35,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:51:35,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.64 >MainThread::INFO::2013-11-14 16:51:35,102::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:35,103::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440695|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440695 (Thu Nov 14 16:51:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.64 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:51:35,103::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:51:35,111::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:51:35,112::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:51:35,112::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440695, 'last-update-local-ts': 1384440695.111635, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:35,112::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:51:35,112::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440689, 'last-update-local-ts': 1384440695.111635, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:35,112::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:51:45,192::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:51:45,446::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:51:45,521::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:51:45,582::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:51:45,582::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:51:45,583::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:51:45,583::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:45,583::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:51:45,583::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:51:45,584::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.545 >MainThread::INFO::2013-11-14 16:51:45,584::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:45,585::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440705|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440705 (Thu Nov 14 16:51:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.545 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:51:45,585::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:51:45,594::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:51:45,594::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:51:45,594::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440705, 'last-update-local-ts': 1384440705.594172, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:45,595::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:51:45,595::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440699, 'last-update-local-ts': 1384440705.594172, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:45,596::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:51:55,672::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:51:55,925::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:51:55,985::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:51:56,047::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:51:56,048::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:51:56,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:51:56,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:56,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 16:51:56,049::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:51:56,050::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.46 >MainThread::INFO::2013-11-14 16:51:56,051::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:51:56,052::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440716|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440716 (Thu Nov 14 16:51:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.46 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:51:56,052::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:51:56,071::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:51:56,072::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:51:56,072::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440716, 'last-update-local-ts': 1384440716.071794, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:56,072::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:51:56,073::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440710, 'last-update-local-ts': 1384440716.071794, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:51:56,073::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:52:06,154::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:52:06,454::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:52:06,559::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:52:06,621::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:52:06,621::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:52:06,622::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:52:06,622::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:06,622::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:52:06,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:52:06,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.39 >MainThread::INFO::2013-11-14 16:52:06,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:06,624::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440726|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440726 (Thu Nov 14 16:52:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.39 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:52:06,624::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:52:06,643::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:52:06,644::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:52:06,644::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440726, 'last-update-local-ts': 1384440726.6437709, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:06,644::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:52:06,644::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440720, 'last-update-local-ts': 1384440726.6437709, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:06,645::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:52:16,719::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:52:16,967::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:52:17,041::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:52:17,105::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:52:17,106::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:52:17,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:52:17,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:17,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:52:17,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:52:17,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.37 >MainThread::INFO::2013-11-14 16:52:17,107::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:17,109::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440737|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440737 (Thu Nov 14 16:52:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.37 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:52:17,109::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:52:17,118::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:52:17,118::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:52:17,118::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440737, 'last-update-local-ts': 1384440737.118305, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:17,119::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:52:17,119::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440731, 'last-update-local-ts': 1384440737.118305, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:17,119::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:52:27,216::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:52:27,465::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:52:27,574::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:52:27,634::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:52:27,635::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:52:27,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:52:27,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:27,635::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:52:27,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:52:27,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 16:52:27,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:27,637::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440747|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440747 (Thu Nov 14 16:52:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:52:27,637::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:52:27,645::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:52:27,646::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:52:27,646::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440747, 'last-update-local-ts': 1384440747.6458571, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:27,646::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:52:27,646::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440741, 'last-update-local-ts': 1384440747.6458571, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:27,647::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:52:37,725::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:52:37,974::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:52:38,083::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:52:38,144::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:52:38,144::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:52:38,144::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:52:38,145::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:38,145::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:52:38,145::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:52:38,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 16:52:38,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:38,147::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440758|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440758 (Thu Nov 14 16:52:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.265 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:52:38,148::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:52:38,156::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:52:38,157::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:52:38,157::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440758, 'last-update-local-ts': 1384440758.156702, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:38,157::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:52:38,157::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440752, 'last-update-local-ts': 1384440758.156702, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:38,158::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:52:48,228::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:52:48,483::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:52:48,569::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:52:48,630::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:52:48,630::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:52:48,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:52:48,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:48,631::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:52:48,632::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:52:48,632::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 16:52:48,632::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:48,632::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440768|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440768 (Thu Nov 14 16:52:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:52:48,633::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:52:48,641::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:52:48,642::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:52:48,642::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440768, 'last-update-local-ts': 1384440768.641649, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:48,642::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:52:48,642::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440762, 'last-update-local-ts': 1384440768.641649, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:48,642::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:52:58,716::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:52:58,969::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:52:59,034::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:52:59,092::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:52:59,093::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:52:59,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:52:59,099::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:59,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:52:59,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:52:59,100::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.28 >MainThread::INFO::2013-11-14 16:52:59,101::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:52:59,101::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440779|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440779 (Thu Nov 14 16:52:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.28 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:52:59,101::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:52:59,111::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:52:59,111::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:52:59,112::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440779, 'last-update-local-ts': 1384440779.1113181, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:59,112::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:52:59,112::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440773, 'last-update-local-ts': 1384440779.1113181, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:52:59,112::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:53:09,194::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:53:09,455::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:53:09,571::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:53:09,647::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:53:09,648::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:53:09,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:53:09,651::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:09,651::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:53:09,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:53:09,658::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 16:53:09,659::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:09,660::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440789|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440789 (Thu Nov 14 16:53:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.32 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:53:09,661::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:53:09,670::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:53:09,670::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:53:09,670::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440789, 'last-update-local-ts': 1384440789.6698959, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:09,670::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:53:09,670::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440783, 'last-update-local-ts': 1384440789.6698959, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:09,671::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:53:19,748::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:53:19,998::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:53:20,085::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:53:20,152::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:53:20,152::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:53:20,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:53:20,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:20,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:53:20,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:53:20,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.27 >MainThread::INFO::2013-11-14 16:53:20,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:20,155::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440800|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440800 (Thu Nov 14 16:53:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.27 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:53:20,155::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:53:20,163::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:53:20,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:53:20,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440800, 'last-update-local-ts': 1384440800.163646, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:20,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:53:20,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440793, 'last-update-local-ts': 1384440800.163646, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:20,164::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:53:30,247::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:53:30,504::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:53:30,598::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:53:30,659::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:53:30,660::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:53:30,660::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:53:30,660::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:30,661::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:53:30,661::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:53:30,661::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 16:53:30,662::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:30,663::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440810|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440810 (Thu Nov 14 16:53:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:53:30,663::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:53:30,672::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:53:30,672::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:53:30,672::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440810, 'last-update-local-ts': 1384440810.671983, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:30,672::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:53:30,673::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440804, 'last-update-local-ts': 1384440810.671983, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:30,673::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:53:40,744::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:53:40,992::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:53:41,059::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:53:41,119::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:53:41,120::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:53:41,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:53:41,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:41,121::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:53:41,122::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:53:41,122::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 16:53:41,122::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:41,123::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440821|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440821 (Thu Nov 14 16:53:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:53:41,123::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:53:41,131::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:53:41,132::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:53:41,132::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440821, 'last-update-local-ts': 1384440821.131686, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:41,132::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:53:41,132::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440814, 'last-update-local-ts': 1384440821.131686, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:41,132::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:53:51,213::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:53:51,466::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:53:51,571::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:53:51,674::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:53:51,674::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:53:51,682::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:53:51,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:51,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:53:51,683::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:53:51,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 16:53:51,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:53:51,685::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440831|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440831 (Thu Nov 14 16:53:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:53:51,685::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:53:51,694::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:53:51,695::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:53:51,695::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440831, 'last-update-local-ts': 1384440831.6947351, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:51,695::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:53:51,695::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440825, 'last-update-local-ts': 1384440831.6947351, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:53:51,696::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:54:01,777::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:54:02,031::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:54:02,116::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:54:02,182::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:54:02,182::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:54:02,183::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:54:02,183::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:02,183::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:54:02,184::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:54:02,184::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 16:54:02,184::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:02,185::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440842|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440842 (Thu Nov 14 16:54:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:54:02,186::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:54:02,199::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:54:02,200::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:54:02,200::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440842, 'last-update-local-ts': 1384440842.199666, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:02,200::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:54:02,200::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440835, 'last-update-local-ts': 1384440842.199666, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:02,201::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:54:12,274::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:54:12,530::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:54:12,626::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:54:12,692::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:54:12,693::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:54:12,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:54:12,693::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:12,694::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:54:12,694::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:54:12,694::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 16:54:12,695::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:12,695::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440852|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440852 (Thu Nov 14 16:54:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:54:12,695::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:54:12,704::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:54:12,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:54:12,704::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440852, 'last-update-local-ts': 1384440852.7039151, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:12,704::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:54:12,704::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440846, 'last-update-local-ts': 1384440852.7039151, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:12,705::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:54:22,785::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:54:23,038::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:54:23,116::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:54:23,176::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:54:23,177::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:54:23,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:54:23,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:23,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:54:23,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:54:23,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 16:54:23,178::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:23,179::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440863|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440863 (Thu Nov 14 16:54:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:54:23,180::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:54:23,189::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:54:23,189::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:54:23,189::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440863, 'last-update-local-ts': 1384440863.1892591, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:23,190::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:54:23,190::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440856, 'last-update-local-ts': 1384440863.1892591, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:23,190::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:54:33,271::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:54:33,540::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:54:33,598::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:54:33,658::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:54:33,658::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:54:33,659::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:54:33,659::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:33,659::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:54:33,659::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:54:33,660::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 16:54:33,660::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:33,660::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440873|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440873 (Thu Nov 14 16:54:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:54:33,661::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:54:33,669::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:54:33,670::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:54:33,670::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440873, 'last-update-local-ts': 1384440873.6696689, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:33,670::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:54:33,670::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440867, 'last-update-local-ts': 1384440873.6696689, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:33,670::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:54:43,745::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:54:43,997::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:54:44,080::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:54:44,152::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:54:44,152::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:54:44,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:54:44,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:44,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:54:44,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:54:44,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 16:54:44,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:44,155::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440884|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440884 (Thu Nov 14 16:54:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:54:44,155::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:54:44,163::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:54:44,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:54:44,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440884, 'last-update-local-ts': 1384440884.163605, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:44,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:54:44,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440877, 'last-update-local-ts': 1384440884.163605, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:44,164::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:54:54,245::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:54:54,497::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:54:54,591::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:54:54,651::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:54:54,652::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:54:54,652::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:54:54,653::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:54,653::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:54:54,653::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:54:54,654::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.395 >MainThread::INFO::2013-11-14 16:54:54,654::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:54:54,655::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440894|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440894 (Thu Nov 14 16:54:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.395 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:54:54,655::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:54:54,666::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:54:54,667::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:54:54,667::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440894, 'last-update-local-ts': 1384440894.6665969, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:54,667::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:54:54,667::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440888, 'last-update-local-ts': 1384440894.6665969, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:54:54,667::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:55:04,764::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:55:05,062::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:55:05,191::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:55:05,251::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:55:05,251::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:55:05,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:55:05,252::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:05,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 16:55:05,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:55:05,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 16:55:05,253::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:05,255::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440905|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440905 (Thu Nov 14 16:55:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:55:05,255::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:55:05,261::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:55:05,262::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:55:05,262::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440905, 'last-update-local-ts': 1384440905.2617691, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:05,262::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:55:05,262::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440898, 'last-update-local-ts': 1384440905.2617691, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:05,263::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:55:15,347::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:55:15,606::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:55:15,658::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:55:15,718::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:55:15,718::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:55:15,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:55:15,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:15,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 16:55:15,719::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:55:15,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.43 >MainThread::INFO::2013-11-14 16:55:15,720::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:15,720::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440915|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440915 (Thu Nov 14 16:55:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.43 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:55:15,721::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:55:15,731::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:55:15,731::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:55:15,731::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440915, 'last-update-local-ts': 1384440915.731235, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:15,732::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:55:15,732::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440909, 'last-update-local-ts': 1384440915.731235, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:15,732::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:55:25,801::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:55:26,056::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:55:26,146::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:55:26,207::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:55:26,207::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:55:26,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:55:26,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:26,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:55:26,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:55:26,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.365 >MainThread::INFO::2013-11-14 16:55:26,209::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:26,209::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440926|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440926 (Thu Nov 14 16:55:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.365 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:55:26,210::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:55:26,218::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:55:26,218::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:55:26,219::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440926, 'last-update-local-ts': 1384440926.2184999, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:26,219::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:55:26,219::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440919, 'last-update-local-ts': 1384440926.2184999, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:26,219::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:55:36,294::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:55:36,556::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:55:36,649::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:55:36,711::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:55:36,712::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:55:36,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:55:36,712::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:36,713::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:55:36,713::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:55:36,713::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.31 >MainThread::INFO::2013-11-14 16:55:36,714::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:36,715::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440936|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440936 (Thu Nov 14 16:55:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.31 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:55:36,715::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:55:36,724::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:55:36,724::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:55:36,724::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440936, 'last-update-local-ts': 1384440936.7242489, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:36,725::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:55:36,725::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440929, 'last-update-local-ts': 1384440936.7242489, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:36,725::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:55:46,802::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:55:47,051::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:55:47,115::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:55:47,174::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:55:47,175::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:55:47,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:55:47,175::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:47,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:55:47,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:55:47,176::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.26 >MainThread::INFO::2013-11-14 16:55:47,177::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:47,177::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440947|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440947 (Thu Nov 14 16:55:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.26 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:55:47,177::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:55:47,185::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:55:47,186::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:55:47,186::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440947, 'last-update-local-ts': 1384440947.185631, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:47,186::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:55:47,186::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440940, 'last-update-local-ts': 1384440947.185631, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:47,186::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:55:57,256::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:55:57,503::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:55:57,586::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:55:57,647::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:55:57,647::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:55:57,648::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:55:57,648::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:57,648::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:55:57,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:55:57,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.22 >MainThread::INFO::2013-11-14 16:55:57,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:55:57,649::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440957|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440957 (Thu Nov 14 16:55:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.22 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:55:57,650::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:55:57,658::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:55:57,659::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:55:57,659::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440957, 'last-update-local-ts': 1384440957.65868, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:57,659::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:55:57,659::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440950, 'last-update-local-ts': 1384440957.65868, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:55:57,660::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:56:07,736::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:56:07,994::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:56:08,104::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:56:08,166::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:56:08,166::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:56:08,166::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:56:08,167::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:08,167::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:56:08,167::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:56:08,167::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 16:56:08,168::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:08,169::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440968|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440968 (Thu Nov 14 16:56:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:56:08,169::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:56:08,178::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:56:08,178::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:56:08,179::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440968, 'last-update-local-ts': 1384440968.1784661, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:08,179::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:56:08,179::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440961, 'last-update-local-ts': 1384440968.1784661, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:08,179::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:56:18,260::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:56:18,508::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:56:18,603::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:56:18,663::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:56:18,664::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:56:18,664::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:56:18,664::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:18,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:56:18,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:56:18,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:56:18,665::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:18,666::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440978|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440978 (Thu Nov 14 16:56:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:56:18,666::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:56:18,674::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:56:18,675::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:56:18,675::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440978, 'last-update-local-ts': 1384440978.6748431, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:18,675::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:56:18,675::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440971, 'last-update-local-ts': 1384440978.6748431, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:18,676::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:56:28,750::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:56:29,002::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:56:29,094::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:56:29,153::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:56:29,153::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:56:29,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:56:29,154::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:29,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 16:56:29,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:56:29,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 16:56:29,155::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:29,157::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440989|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440989 (Thu Nov 14 16:56:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:56:29,157::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:56:29,167::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:56:29,167::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:56:29,168::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440989, 'last-update-local-ts': 1384440989.167377, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:29,168::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:56:29,168::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440982, 'last-update-local-ts': 1384440989.167377, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:29,168::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:56:39,251::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:56:39,498::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:56:39,552::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:56:39,611::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:56:39,611::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:56:39,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:56:39,612::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:39,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:56:39,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:56:39,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 16:56:39,613::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:39,614::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384440999|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384440999 (Thu Nov 14 16:56:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:56:39,614::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:56:39,629::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:56:39,630::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:56:39,630::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384440999, 'last-update-local-ts': 1384440999.62988, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:39,630::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:56:39,630::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384440992, 'last-update-local-ts': 1384440999.62988, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:39,631::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:56:49,727::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:56:49,986::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:56:50,079::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:56:50,139::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:56:50,139::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:56:50,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:56:50,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:50,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:56:50,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:56:50,140::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 16:56:50,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:56:50,141::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441010|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441010 (Thu Nov 14 16:56:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:56:50,141::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:56:50,148::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:56:50,148::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:56:50,149::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441010, 'last-update-local-ts': 1384441010.1483741, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:50,149::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:56:50,149::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441003, 'last-update-local-ts': 1384441010.1483741, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:56:50,149::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:57:00,230::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:57:00,488::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:57:00,587::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:57:00,647::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:57:00,647::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:57:00,648::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:57:00,648::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:00,648::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:57:00,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:57:00,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 16:57:00,649::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:00,650::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441020|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441020 (Thu Nov 14 16:57:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:57:00,651::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:57:00,658::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:57:00,659::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:57:00,659::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441020, 'last-update-local-ts': 1384441020.6588249, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:00,659::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:57:00,659::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441013, 'last-update-local-ts': 1384441020.6588249, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:00,660::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:57:10,751::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:57:11,008::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:57:11,085::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:57:11,147::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:57:11,148::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:57:11,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:57:11,148::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:11,149::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:57:11,149::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:57:11,149::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.18 >MainThread::INFO::2013-11-14 16:57:11,150::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:11,150::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441031|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441031 (Thu Nov 14 16:57:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.18 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:57:11,150::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:57:11,158::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:57:11,158::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:57:11,158::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441031, 'last-update-local-ts': 1384441031.1579001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:11,158::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:57:11,158::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441024, 'last-update-local-ts': 1384441031.1579001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:11,159::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:57:21,232::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:57:21,486::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:57:21,576::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:57:21,636::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:57:21,636::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:57:21,643::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:57:21,643::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:21,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:57:21,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:57:21,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 16:57:21,644::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:21,645::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441041|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441041 (Thu Nov 14 16:57:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:57:21,646::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:57:21,654::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:57:21,654::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:57:21,654::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441041, 'last-update-local-ts': 1384441041.6543069, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:21,655::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:57:21,655::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441034, 'last-update-local-ts': 1384441041.6543069, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:21,655::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:57:31,739::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:57:31,991::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:57:32,044::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:57:32,104::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:57:32,105::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:57:32,105::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:57:32,105::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:32,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:57:32,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:57:32,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 16:57:32,106::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:32,107::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441052|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441052 (Thu Nov 14 16:57:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:57:32,107::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:57:32,116::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:57:32,116::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:57:32,116::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441052, 'last-update-local-ts': 1384441052.116087, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:32,116::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:57:32,117::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441044, 'last-update-local-ts': 1384441052.116087, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:32,117::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:57:42,195::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:57:42,471::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:57:42,567::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:57:42,628::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:57:42,629::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:57:42,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:57:42,629::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:42,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:57:42,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:57:42,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.15 >MainThread::INFO::2013-11-14 16:57:42,630::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:42,631::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441062|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441062 (Thu Nov 14 16:57:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.15 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:57:42,631::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:57:42,640::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:57:42,640::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:57:42,640::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441062, 'last-update-local-ts': 1384441062.6399679, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:42,640::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:57:42,641::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441055, 'last-update-local-ts': 1384441062.6399679, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:42,641::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:57:52,717::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:57:52,991::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:57:53,080::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:57:53,140::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:57:53,140::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:57:53,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:57:53,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:53,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:57:53,141::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:57:53,142::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 16:57:53,142::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:57:53,143::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441073|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441073 (Thu Nov 14 16:57:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:57:53,143::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:57:53,150::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:57:53,151::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:57:53,151::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441073, 'last-update-local-ts': 1384441073.15083, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:53,151::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:57:53,151::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441065, 'last-update-local-ts': 1384441073.15083, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:57:53,152::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:58:03,256::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:58:03,508::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:58:03,593::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:58:03,660::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:58:03,660::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:58:03,661::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:58:03,661::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:03,662::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:58:03,662::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:58:03,662::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 16:58:03,662::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:03,663::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441083|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441083 (Thu Nov 14 16:58:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:58:03,663::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:58:03,672::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:58:03,673::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:58:03,673::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441083, 'last-update-local-ts': 1384441083.672627, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:03,673::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:58:03,673::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441076, 'last-update-local-ts': 1384441083.672627, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:03,673::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:58:13,795::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:58:14,109::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:58:14,201::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:58:14,262::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:58:14,263::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:58:14,269::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:58:14,269::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:14,270::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:58:14,270::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:58:14,270::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 16:58:14,270::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:14,271::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441094|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441094 (Thu Nov 14 16:58:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:58:14,271::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:58:14,281::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:58:14,282::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:58:14,282::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441094, 'last-update-local-ts': 1384441094.2817481, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:14,282::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:58:14,282::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441086, 'last-update-local-ts': 1384441094.2817481, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:14,283::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:58:24,366::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:58:24,618::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:58:24,678::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:58:24,738::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:58:24,738::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:58:24,739::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:58:24,739::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:24,739::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:58:24,740::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:58:24,740::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 16:58:24,740::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:24,741::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441104|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441104 (Thu Nov 14 16:58:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:58:24,741::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:58:24,750::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:58:24,750::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:58:24,750::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441104, 'last-update-local-ts': 1384441104.750248, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:24,751::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:58:24,751::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441097, 'last-update-local-ts': 1384441104.750248, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:24,751::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:58:34,840::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:58:35,112::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:58:35,195::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:58:35,262::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:58:35,263::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:58:35,263::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:58:35,263::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:35,264::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7035 >MainThread::INFO::2013-11-14 16:58:35,264::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:58:35,264::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 16:58:35,264::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:35,265::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441115|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441115 (Thu Nov 14 16:58:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7035 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:58:35,265::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:58:35,272::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:58:35,272::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:58:35,272::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441115, 'last-update-local-ts': 1384441115.271894, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:35,272::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:58:35,272::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441107, 'last-update-local-ts': 1384441115.271894, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:35,273::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:58:45,360::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:58:45,616::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:58:45,700::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:58:45,761::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:58:45,761::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:58:45,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:58:45,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:45,762::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:58:45,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:58:45,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 16:58:45,763::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:45,764::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441125|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441125 (Thu Nov 14 16:58:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:58:45,764::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:58:45,772::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:58:45,772::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:58:45,772::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441125, 'last-update-local-ts': 1384441125.7719259, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:45,772::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:58:45,772::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441118, 'last-update-local-ts': 1384441125.7719259, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:45,773::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:58:55,866::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:58:56,115::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:58:56,195::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:58:56,254::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:58:56,255::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:58:56,255::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:58:56,255::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:56,256::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 16:58:56,256::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:58:56,256::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 16:58:56,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:58:56,258::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441136|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441136 (Thu Nov 14 16:58:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:58:56,258::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:58:56,267::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:58:56,267::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:58:56,267::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441136, 'last-update-local-ts': 1384441136.266969, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:56,267::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:58:56,268::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441128, 'last-update-local-ts': 1384441136.266969, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:58:56,268::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:59:06,347::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:59:06,602::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:59:06,671::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:59:06,732::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:59:06,732::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:59:06,733::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:59:06,733::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:06,733::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:59:06,734::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:59:06,734::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 16:59:06,734::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:06,735::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441146|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441146 (Thu Nov 14 16:59:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:59:06,735::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:59:06,744::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:59:06,744::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:59:06,745::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441146, 'last-update-local-ts': 1384441146.7443891, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:06,745::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:59:06,745::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441149, 'last-update-local-ts': 1384441146.7443891, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:06,745::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:59:16,832::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:59:17,086::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:59:17,181::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:59:17,241::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:59:17,241::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:59:17,242::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:59:17,242::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:17,242::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:59:17,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:59:17,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 16:59:17,243::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:17,244::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441157|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441157 (Thu Nov 14 16:59:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:59:17,244::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:59:17,252::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:59:17,253::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:59:17,253::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441157, 'last-update-local-ts': 1384441157.2527311, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:17,253::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:59:17,253::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441159, 'last-update-local-ts': 1384441157.2527311, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:17,254::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:59:27,345::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:59:27,594::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:59:27,672::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:59:27,733::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:59:27,734::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:59:27,734::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:59:27,734::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:27,735::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:59:27,735::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:59:27,735::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.29 >MainThread::INFO::2013-11-14 16:59:27,736::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:27,737::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441167|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441167 (Thu Nov 14 16:59:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.29 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:59:27,737::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:59:27,745::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:59:27,745::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:59:27,746::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441167, 'last-update-local-ts': 1384441167.7455039, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:27,746::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:59:27,746::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441170, 'last-update-local-ts': 1384441167.7455039, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:27,746::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:59:37,821::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:59:38,075::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:59:38,164::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:59:38,224::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:59:38,225::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:59:38,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:59:38,231::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:38,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 16:59:38,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:59:38,232::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 16:59:38,233::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:38,233::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441178|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441178 (Thu Nov 14 16:59:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:59:38,233::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:59:38,242::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:59:38,242::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:59:38,242::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441178, 'last-update-local-ts': 1384441178.242306, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:38,243::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:59:38,243::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441180, 'last-update-local-ts': 1384441178.242306, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:38,243::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:59:48,315::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:59:48,569::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:59:48,625::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:59:48,683::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:59:48,684::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:59:48,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:59:48,684::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:48,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:59:48,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:59:48,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 16:59:48,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:48,687::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441188|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441188 (Thu Nov 14 16:59:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:59:48,687::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:59:48,695::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:59:48,696::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:59:48,696::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441188, 'last-update-local-ts': 1384441188.695658, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:48,696::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:59:48,696::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441191, 'last-update-local-ts': 1384441188.695658, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:48,696::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 16:59:58,791::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 16:59:59,051::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 16:59:59,142::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 16:59:59,202::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 16:59:59,203::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 16:59:59,203::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 16:59:59,203::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:59,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 16:59:59,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 16:59:59,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.685 >MainThread::INFO::2013-11-14 16:59:59,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 16:59:59,205::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441199|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441199 (Thu Nov 14 16:59:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.685 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 16:59:59,205::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 16:59:59,212::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 16:59:59,213::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 16:59:59,213::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441199, 'last-update-local-ts': 1384441199.2126491, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:59,213::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 16:59:59,213::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441201, 'last-update-local-ts': 1384441199.2126491, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 16:59:59,213::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:00:09,315::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:00:09,578::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:00:09,754::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:00:09,816::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:00:09,816::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:00:09,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:00:09,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:09,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7032 >MainThread::INFO::2013-11-14 17:00:09,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:00:09,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.58 >MainThread::INFO::2013-11-14 17:00:09,818::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:09,819::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441209|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441209 (Thu Nov 14 17:00:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.58 >engine-health=vm-down >gateway=True >mem-free=7032 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:00:09,820::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:00:09,828::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:00:09,828::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:00:09,828::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441209, 'last-update-local-ts': 1384441209.828217, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:09,829::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:00:09,829::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441212, 'last-update-local-ts': 1384441209.828217, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:09,829::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:00:19,930::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:00:20,187::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:00:20,297::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:00:20,358::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:00:20,358::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:00:20,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:00:20,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:20,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:00:20,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:00:20,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.53 >MainThread::INFO::2013-11-14 17:00:20,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:20,361::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441220|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441220 (Thu Nov 14 17:00:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.53 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:00:20,361::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:00:20,369::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:00:20,369::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:00:20,370::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441220, 'last-update-local-ts': 1384441220.3694589, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:20,370::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:00:20,370::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441222, 'last-update-local-ts': 1384441220.3694589, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:20,370::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:00:30,459::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:00:30,708::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:00:30,778::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:00:30,839::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:00:30,839::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:00:30,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:00:30,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:30,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:00:30,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:00:30,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.485 >MainThread::INFO::2013-11-14 17:00:30,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:30,842::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441230|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441230 (Thu Nov 14 17:00:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.485 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:00:30,842::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:00:30,851::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:00:30,851::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:00:30,852::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441230, 'last-update-local-ts': 1384441230.8513601, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:30,852::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:00:30,852::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441233, 'last-update-local-ts': 1384441230.8513601, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:30,852::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:00:40,936::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:00:41,190::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:00:41,272::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:00:41,333::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:00:41,334::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:00:41,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:00:41,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:41,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:00:41,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:00:41,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.41 >MainThread::INFO::2013-11-14 17:00:41,342::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:41,342::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441241|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441241 (Thu Nov 14 17:00:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.41 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:00:41,342::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:00:41,352::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:00:41,352::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:00:41,352::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441241, 'last-update-local-ts': 1384441241.3520579, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:41,352::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:00:41,353::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441243, 'last-update-local-ts': 1384441241.3520579, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:41,353::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:00:51,424::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:00:51,694::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:00:51,751::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:00:51,811::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:00:51,811::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:00:51,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:00:51,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:51,812::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:00:51,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:00:51,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.345 >MainThread::INFO::2013-11-14 17:00:51,813::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:00:51,814::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441251|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441251 (Thu Nov 14 17:00:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.345 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:00:51,814::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:00:51,823::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:00:51,823::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:00:51,823::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441251, 'last-update-local-ts': 1384441251.823267, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:51,824::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:00:51,824::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441253, 'last-update-local-ts': 1384441251.823267, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:00:51,824::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:01:01,943::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:01:02,233::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:01:02,371::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:01:02,435::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:01:02,436::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:01:02,436::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:01:02,437::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:02,437::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:01:02,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:01:02,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.245 >MainThread::INFO::2013-11-14 17:01:02,438::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:02,439::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441262|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441262 (Thu Nov 14 17:01:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.245 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:01:02,439::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:01:02,452::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:01:02,453::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:01:02,453::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441262, 'last-update-local-ts': 1384441262.452683, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:02,453::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:01:02,453::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441264, 'last-update-local-ts': 1384441262.452683, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:02,454::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:01:12,565::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:01:12,823::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:01:12,950::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:01:13,017::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:01:13,018::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:01:13,018::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0015243902439 >MainThread::INFO::2013-11-14 17:01:13,018::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:13,019::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6942 >MainThread::INFO::2013-11-14 17:01:13,019::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:01:13,019::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.445 >MainThread::INFO::2013-11-14 17:01:13,020::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:13,020::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441273|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441273 (Thu Nov 14 17:01:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.445 >engine-health=vm-down >gateway=True >mem-free=6942 >mem-load=0.0015243902439 > >MainThread::INFO::2013-11-14 17:01:13,020::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:01:13,028::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:01:13,029::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:01:13,029::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441273, 'last-update-local-ts': 1384441273.0287981, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:13,029::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:01:13,029::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441274, 'last-update-local-ts': 1384441273.0287981, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:13,030::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:01:23,108::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:01:23,360::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:01:23,461::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:01:23,522::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:01:23,522::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:01:23,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:01:23,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:23,529::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:01:23,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:01:23,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.415 >MainThread::INFO::2013-11-14 17:01:23,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:23,531::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441283|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441283 (Thu Nov 14 17:01:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.415 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:01:23,531::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:01:23,539::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:01:23,539::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:01:23,540::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441283, 'last-update-local-ts': 1384441283.539412, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:23,540::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:01:23,540::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441285, 'last-update-local-ts': 1384441283.539412, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:23,540::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:01:33,609::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:01:33,861::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:01:33,915::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:01:33,975::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:01:33,976::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:01:33,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:01:33,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:33,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:01:33,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:01:33,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.35 >MainThread::INFO::2013-11-14 17:01:33,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:33,978::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441293|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441293 (Thu Nov 14 17:01:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.35 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:01:33,978::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:01:33,987::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:01:33,987::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:01:33,987::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441293, 'last-update-local-ts': 1384441293.98701, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:33,987::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:01:33,988::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441295, 'last-update-local-ts': 1384441293.98701, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:33,988::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:01:44,064::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:01:44,316::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:01:44,398::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:01:44,458::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:01:44,459::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:01:44,459::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:01:44,459::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:44,460::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:01:44,460::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:01:44,460::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.375 >MainThread::INFO::2013-11-14 17:01:44,460::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:44,461::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441304|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441304 (Thu Nov 14 17:01:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.375 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:01:44,461::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:01:44,469::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:01:44,470::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:01:44,470::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441304, 'last-update-local-ts': 1384441304.469728, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:44,470::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:01:44,470::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441306, 'last-update-local-ts': 1384441304.469728, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:44,471::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:01:54,545::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:01:54,800::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:01:54,897::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:01:54,957::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:01:54,958::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:01:54,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:01:54,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:54,959::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 17:01:54,959::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:01:54,959::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.315 >MainThread::INFO::2013-11-14 17:01:54,960::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:01:54,961::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441314|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441314 (Thu Nov 14 17:01:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.315 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:01:54,961::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:01:54,969::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:01:54,969::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:01:54,970::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441314, 'last-update-local-ts': 1384441314.9693279, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:54,970::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:01:54,970::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441316, 'last-update-local-ts': 1384441314.9693279, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:01:54,970::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:02:05,050::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:02:05,326::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:02:05,409::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:02:05,469::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:02:05,469::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:02:05,470::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:02:05,470::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:05,471::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7037 >MainThread::INFO::2013-11-14 17:02:05,471::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:02:05,471::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 17:02:05,471::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:05,472::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441325|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441325 (Thu Nov 14 17:02:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.265 >engine-health=vm-down >gateway=True >mem-free=7037 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:02:05,472::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:02:05,478::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:02:05,479::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:02:05,479::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441325, 'last-update-local-ts': 1384441325.4788401, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:05,479::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:02:05,479::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441327, 'last-update-local-ts': 1384441325.4788401, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:05,480::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:02:15,553::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:02:15,825::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:02:15,889::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:02:15,949::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:02:15,949::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:02:15,950::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:02:15,950::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:15,950::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 17:02:15,951::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:02:15,951::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 17:02:15,951::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:15,952::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441335|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441335 (Thu Nov 14 17:02:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:02:15,952::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:02:15,960::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:02:15,960::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:02:15,960::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441335, 'last-update-local-ts': 1384441335.9599681, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:15,960::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:02:15,960::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441337, 'last-update-local-ts': 1384441335.9599681, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:15,961::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:02:26,060::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:02:26,309::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:02:26,400::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:02:26,460::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:02:26,461::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:02:26,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:02:26,461::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:26,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 17:02:26,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:02:26,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 17:02:26,462::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:26,463::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441346|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441346 (Thu Nov 14 17:02:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:02:26,463::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:02:26,471::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:02:26,472::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:02:26,472::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441346, 'last-update-local-ts': 1384441346.471796, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:26,472::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:02:26,472::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441348, 'last-update-local-ts': 1384441346.471796, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:26,473::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:02:36,548::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:02:36,799::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:02:36,891::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:02:36,969::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:02:36,969::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:02:36,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:02:36,976::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:36,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 17:02:36,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:02:36,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 17:02:36,977::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:36,978::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441356|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441356 (Thu Nov 14 17:02:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.32 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:02:36,978::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:02:36,988::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:02:36,988::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:02:36,988::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441356, 'last-update-local-ts': 1384441356.988106, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:36,988::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:02:36,989::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441358, 'last-update-local-ts': 1384441356.988106, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:36,989::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:02:47,070::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:02:47,326::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:02:47,406::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:02:47,466::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:02:47,466::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:02:47,466::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:02:47,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:47,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:02:47,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:02:47,467::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.27 >MainThread::INFO::2013-11-14 17:02:47,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:47,469::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441367|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441367 (Thu Nov 14 17:02:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.27 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:02:47,469::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:02:47,477::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:02:47,478::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:02:47,478::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441367, 'last-update-local-ts': 1384441367.477798, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:47,478::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:02:47,478::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441368, 'last-update-local-ts': 1384441367.477798, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:47,479::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:02:57,555::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:02:57,813::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:02:57,895::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:02:57,957::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:02:57,957::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:02:57,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:02:57,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:57,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:02:57,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:02:57,959::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 17:02:57,959::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:02:57,959::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441377|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441377 (Thu Nov 14 17:02:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:02:57,960::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:02:57,968::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:02:57,968::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:02:57,969::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441377, 'last-update-local-ts': 1384441377.9684131, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:57,969::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:02:57,969::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441379, 'last-update-local-ts': 1384441377.9684131, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:02:57,969::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:03:08,037::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:03:08,291::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:03:08,351::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:03:08,410::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:03:08,411::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:03:08,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:03:08,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:08,418::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:03:08,418::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:03:08,418::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 17:03:08,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:08,419::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441388|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441388 (Thu Nov 14 17:03:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:03:08,419::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:03:08,429::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:03:08,430::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:03:08,430::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441388, 'last-update-local-ts': 1384441388.4296169, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:08,430::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:03:08,430::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441389, 'last-update-local-ts': 1384441388.4296169, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:08,430::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:03:18,519::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:03:18,769::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:03:18,872::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:03:18,932::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:03:18,933::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:03:18,933::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:03:18,933::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:18,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:03:18,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:03:18,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 17:03:18,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:18,936::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441398|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441398 (Thu Nov 14 17:03:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.32 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:03:18,936::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:03:18,944::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:03:18,944::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:03:18,945::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441398, 'last-update-local-ts': 1384441398.9444301, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:18,945::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:03:18,945::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441400, 'last-update-local-ts': 1384441398.9444301, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:18,945::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:03:29,022::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:03:29,270::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:03:29,365::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:03:29,438::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:03:29,438::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:03:29,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:03:29,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:29,439::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:03:29,440::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:03:29,440::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.305 >MainThread::INFO::2013-11-14 17:03:29,440::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:29,441::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441409|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441409 (Thu Nov 14 17:03:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.305 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:03:29,441::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:03:29,449::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:03:29,450::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:03:29,450::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441409, 'last-update-local-ts': 1384441409.4495871, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:29,450::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:03:29,450::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441410, 'last-update-local-ts': 1384441409.4495871, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:29,450::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:03:39,532::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:03:39,786::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:03:39,861::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:03:39,923::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:03:39,923::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:03:39,924::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:03:39,924::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:39,924::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:03:39,925::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:03:39,925::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.26 >MainThread::INFO::2013-11-14 17:03:39,925::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:39,926::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441419|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441419 (Thu Nov 14 17:03:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.26 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:03:39,926::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:03:39,933::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:03:39,933::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:03:39,934::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441419, 'last-update-local-ts': 1384441419.933373, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:39,934::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:03:39,934::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441421, 'last-update-local-ts': 1384441419.933373, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:39,934::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:03:50,019::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:03:50,275::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:03:50,356::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:03:50,417::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:03:50,418::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:03:50,418::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:03:50,418::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:50,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:03:50,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:03:50,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 17:03:50,419::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:03:50,420::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441430|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441430 (Thu Nov 14 17:03:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:03:50,420::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:03:50,429::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:03:50,429::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:03:50,429::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441430, 'last-update-local-ts': 1384441430.4290209, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:50,429::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:03:50,430::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441431, 'last-update-local-ts': 1384441430.4290209, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:03:50,430::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:04:00,501::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:04:00,761::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:04:00,827::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:04:00,887::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:04:00,887::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:04:00,894::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:04:00,894::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:00,894::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:04:00,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:04:00,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 17:04:00,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:00,896::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441440|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441440 (Thu Nov 14 17:04:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:04:00,896::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:04:00,904::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:04:00,904::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:04:00,904::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441440, 'last-update-local-ts': 1384441440.904171, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:00,905::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:04:00,905::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441442, 'last-update-local-ts': 1384441440.904171, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:00,905::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:04:10,991::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:04:11,243::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:04:11,337::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:04:11,397::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:04:11,398::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:04:11,398::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:04:11,398::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:11,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:04:11,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:04:11,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 17:04:11,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:11,400::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441451|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441451 (Thu Nov 14 17:04:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:04:11,401::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:04:11,407::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:04:11,408::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:04:11,408::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441451, 'last-update-local-ts': 1384441451.4076459, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:11,408::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:04:11,408::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441452, 'last-update-local-ts': 1384441451.4076459, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:11,408::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:04:21,480::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:04:21,730::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:04:21,821::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:04:21,881::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:04:21,882::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:04:21,882::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:04:21,882::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:21,883::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:04:21,883::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:04:21,883::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 17:04:21,883::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:21,884::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441461|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441461 (Thu Nov 14 17:04:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:04:21,884::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:04:21,891::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:04:21,891::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:04:21,891::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441461, 'last-update-local-ts': 1384441461.891115, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:21,891::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:04:21,892::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441462, 'last-update-local-ts': 1384441461.891115, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:21,892::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:04:31,967::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:04:32,237::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:04:32,324::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:04:32,385::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:04:32,385::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:04:32,391::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:04:32,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:32,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:04:32,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:04:32,392::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 17:04:32,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:32,394::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441472|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441472 (Thu Nov 14 17:04:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:04:32,394::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:04:32,402::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:04:32,402::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:04:32,402::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441472, 'last-update-local-ts': 1384441472.402288, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:32,403::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:04:32,403::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441473, 'last-update-local-ts': 1384441472.402288, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:32,403::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:04:42,484::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:04:42,744::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:04:42,821::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:04:42,887::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:04:42,887::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:04:42,888::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:04:42,888::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:42,888::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:04:42,889::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:04:42,889::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.23 >MainThread::INFO::2013-11-14 17:04:42,889::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:42,890::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441482|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441482 (Thu Nov 14 17:04:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.23 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:04:42,890::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:04:42,898::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:04:42,899::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:04:42,899::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441482, 'last-update-local-ts': 1384441482.8986549, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:42,899::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:04:42,899::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441483, 'last-update-local-ts': 1384441482.8986549, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:42,899::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:04:52,979::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:04:53,230::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:04:53,284::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:04:53,344::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:04:53,345::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:04:53,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:04:53,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:53,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:04:53,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:04:53,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.195 >MainThread::INFO::2013-11-14 17:04:53,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:04:53,353::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441493|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441493 (Thu Nov 14 17:04:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.195 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:04:53,353::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:04:53,362::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:04:53,363::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:04:53,363::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441493, 'last-update-local-ts': 1384441493.3625979, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:53,363::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:04:53,363::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441494, 'last-update-local-ts': 1384441493.3625979, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:04:53,363::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:05:03,446::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:05:03,704::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:05:03,802::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:05:03,863::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:05:03,864::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:05:03,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:05:03,865::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:03,865::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:05:03,865::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:05:03,865::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 17:05:03,866::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:03,867::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441503|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441503 (Thu Nov 14 17:05:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:05:03,867::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:05:03,876::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:05:03,876::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:05:03,876::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441503, 'last-update-local-ts': 1384441503.8761079, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:03,876::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:05:03,877::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441504, 'last-update-local-ts': 1384441503.8761079, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:03,877::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:05:13,948::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:05:14,205::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:05:14,295::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:05:14,356::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:05:14,356::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:05:14,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:05:14,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:14,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:05:14,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:05:14,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 17:05:14,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:14,359::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441514|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441514 (Thu Nov 14 17:05:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:05:14,359::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:05:14,367::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:05:14,367::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:05:14,368::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441514, 'last-update-local-ts': 1384441514.367332, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:14,368::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:05:14,368::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441515, 'last-update-local-ts': 1384441514.367332, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:14,368::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:05:24,442::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:05:24,695::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:05:24,784::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:05:24,844::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:05:24,844::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:05:24,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:05:24,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:24,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:05:24,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:05:24,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 17:05:24,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:24,853::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441524|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441524 (Thu Nov 14 17:05:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:05:24,854::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:05:24,863::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:05:24,864::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:05:24,864::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441524, 'last-update-local-ts': 1384441524.8636191, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:24,864::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:05:24,864::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441525, 'last-update-local-ts': 1384441524.8636191, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:24,864::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:05:34,949::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:05:35,202::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:05:35,258::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:05:35,318::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:05:35,318::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:05:35,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:05:35,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:35,319::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7031 >MainThread::INFO::2013-11-14 17:05:35,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:05:35,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 17:05:35,320::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:35,321::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441535|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441535 (Thu Nov 14 17:05:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7031 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:05:35,321::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:05:35,327::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:05:35,328::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:05:35,328::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441535, 'last-update-local-ts': 1384441535.3278191, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:35,328::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:05:35,328::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441536, 'last-update-local-ts': 1384441535.3278191, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:35,329::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:05:45,423::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:05:45,682::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:05:45,763::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:05:45,824::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:05:45,825::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:05:45,825::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:05:45,826::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:45,826::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:05:45,826::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:05:45,826::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 17:05:45,827::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:45,827::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441545|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441545 (Thu Nov 14 17:05:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:05:45,827::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:05:45,834::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:05:45,834::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:05:45,834::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441545, 'last-update-local-ts': 1384441545.834275, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:45,835::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:05:45,835::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441546, 'last-update-local-ts': 1384441545.834275, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:45,835::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:05:55,921::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:05:56,174::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:05:56,264::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:05:56,324::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:05:56,324::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:05:56,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:05:56,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:56,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:05:56,326::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:05:56,326::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.66 >MainThread::INFO::2013-11-14 17:05:56,326::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:05:56,326::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441556|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441556 (Thu Nov 14 17:05:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.66 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:05:56,327::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:05:56,333::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:05:56,334::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:05:56,334::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441556, 'last-update-local-ts': 1384441556.333565, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:56,334::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:05:56,334::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441556, 'last-update-local-ts': 1384441556.333565, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:05:56,334::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:06:06,428::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:06:06,675::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:06:06,753::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:06:06,814::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:06:06,815::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:06:06,815::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:06:06,815::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:06,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:06:06,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:06:06,816::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.56 >MainThread::INFO::2013-11-14 17:06:06,817::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:06,818::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441566|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441566 (Thu Nov 14 17:06:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.56 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:06:06,818::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:06:06,826::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:06:06,827::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:06:06,827::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441566, 'last-update-local-ts': 1384441566.826812, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:06,827::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:06:06,827::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441567, 'last-update-local-ts': 1384441566.826812, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:06,828::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:06:16,903::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:06:17,169::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:06:17,278::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:06:17,373::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:06:17,374::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:06:17,381::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:06:17,382::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:17,382::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:06:17,382::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:06:17,383::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.47 >MainThread::INFO::2013-11-14 17:06:17,383::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:17,383::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441577|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441577 (Thu Nov 14 17:06:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.47 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:06:17,383::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:06:17,394::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:06:17,395::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:06:17,395::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441577, 'last-update-local-ts': 1384441577.3945789, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:17,395::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:06:17,395::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441577, 'last-update-local-ts': 1384441577.3945789, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:17,396::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:06:27,509::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:06:27,781::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:06:27,836::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:06:27,905::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:06:27,905::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:06:27,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:06:27,906::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:27,907::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:06:27,907::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:06:27,907::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.4 >MainThread::INFO::2013-11-14 17:06:27,908::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:27,909::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441587|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441587 (Thu Nov 14 17:06:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.4 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:06:27,910::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:06:27,919::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:06:27,919::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:06:27,919::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441587, 'last-update-local-ts': 1384441587.919095, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:27,920::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:06:27,920::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441588, 'last-update-local-ts': 1384441587.919095, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:27,920::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:06:37,995::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:06:38,244::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:06:38,299::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:06:38,360::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:06:38,360::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:06:38,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:06:38,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:38,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:06:38,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:06:38,362::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.45 >MainThread::INFO::2013-11-14 17:06:38,362::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:38,362::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441598|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441598 (Thu Nov 14 17:06:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.45 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:06:38,363::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:06:38,371::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:06:38,371::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:06:38,372::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441598, 'last-update-local-ts': 1384441598.37147, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:38,372::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:06:38,372::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441598, 'last-update-local-ts': 1384441598.37147, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:38,372::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:06:48,466::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:06:48,739::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:06:48,821::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:06:48,888::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:06:48,888::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:06:48,889::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:06:48,889::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:48,889::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:06:48,890::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:06:48,890::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.455 >MainThread::INFO::2013-11-14 17:06:48,890::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:48,891::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441608|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441608 (Thu Nov 14 17:06:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.455 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:06:48,891::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:06:48,899::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:06:48,900::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:06:48,900::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441608, 'last-update-local-ts': 1384441608.899662, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:48,900::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:06:48,900::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441609, 'last-update-local-ts': 1384441608.899662, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:48,900::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:06:58,978::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:06:59,232::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:06:59,321::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:06:59,383::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:06:59,383::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:06:59,383::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:06:59,384::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:59,384::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:06:59,384::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:06:59,384::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.385 >MainThread::INFO::2013-11-14 17:06:59,385::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:06:59,386::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441619|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441619 (Thu Nov 14 17:06:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.385 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:06:59,386::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:06:59,395::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:06:59,395::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:06:59,395::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441619, 'last-update-local-ts': 1384441619.3949671, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:59,395::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:06:59,396::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441619, 'last-update-local-ts': 1384441619.3949671, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:06:59,396::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:07:09,494::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:07:09,743::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:07:09,821::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:07:09,883::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:07:09,884::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:07:09,884::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:07:09,884::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:09,885::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:07:09,885::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:07:09,885::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.325 >MainThread::INFO::2013-11-14 17:07:09,886::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:09,886::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441629|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441629 (Thu Nov 14 17:07:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.325 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:07:09,886::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:07:09,895::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:07:09,895::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:07:09,895::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441629, 'last-update-local-ts': 1384441629.894999, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:09,895::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:07:09,896::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441630, 'last-update-local-ts': 1384441629.894999, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:09,896::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:07:19,968::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:07:20,263::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:07:20,349::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:07:20,409::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:07:20,409::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:07:20,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:07:20,416::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:20,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:07:20,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:07:20,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.275 >MainThread::INFO::2013-11-14 17:07:20,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:20,418::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441640|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441640 (Thu Nov 14 17:07:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.275 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:07:20,418::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:07:20,426::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:07:20,427::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:07:20,427::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441640, 'last-update-local-ts': 1384441640.4268539, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:20,427::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:07:20,427::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441640, 'last-update-local-ts': 1384441640.4268539, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:20,428::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:07:30,502::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:07:30,756::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:07:30,846::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:07:30,911::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:07:30,911::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:07:30,912::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:07:30,912::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:30,912::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:07:30,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:07:30,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.495 >MainThread::INFO::2013-11-14 17:07:30,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:30,914::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441650|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441650 (Thu Nov 14 17:07:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.495 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:07:30,915::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:07:30,923::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:07:30,924::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:07:30,924::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441650, 'last-update-local-ts': 1384441650.923568, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:30,924::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:07:30,924::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441651, 'last-update-local-ts': 1384441650.923568, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:30,924::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:07:41,014::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:07:41,272::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:07:41,351::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:07:41,412::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:07:41,412::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:07:41,412::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:07:41,413::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:41,413::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:07:41,413::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:07:41,414::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.42 >MainThread::INFO::2013-11-14 17:07:41,414::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:41,414::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441661|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441661 (Thu Nov 14 17:07:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.42 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:07:41,414::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:07:41,423::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:07:41,423::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:07:41,423::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441661, 'last-update-local-ts': 1384441661.4232399, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:41,424::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:07:41,424::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441661, 'last-update-local-ts': 1384441661.4232399, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:41,424::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:07:51,510::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:07:51,768::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:07:51,850::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:07:51,910::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:07:51,911::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:07:51,917::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:07:51,917::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:51,918::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:07:51,918::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:07:51,918::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.355 >MainThread::INFO::2013-11-14 17:07:51,919::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:07:51,919::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441671|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441671 (Thu Nov 14 17:07:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.355 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:07:51,919::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:07:51,929::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:07:51,929::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:07:51,930::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441671, 'last-update-local-ts': 1384441671.9294889, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:51,930::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:07:51,930::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441671, 'last-update-local-ts': 1384441671.9294889, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:07:51,930::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:08:01,996::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:08:02,246::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:08:02,301::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:08:02,360::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:08:02,360::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:08:02,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:08:02,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:02,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:08:02,362::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:08:02,362::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 17:08:02,362::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:02,363::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441682|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441682 (Thu Nov 14 17:08:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:08:02,363::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:08:02,373::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:08:02,373::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:08:02,373::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441682, 'last-update-local-ts': 1384441682.373292, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:02,374::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:08:02,374::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441682, 'last-update-local-ts': 1384441682.373292, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:02,374::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:08:12,475::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:08:12,726::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:08:12,812::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:08:12,881::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:08:12,881::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:08:12,882::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:08:12,882::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:12,882::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:08:12,883::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:08:12,883::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.66 >MainThread::INFO::2013-11-14 17:08:12,883::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:12,884::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441692|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441692 (Thu Nov 14 17:08:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.66 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:08:12,884::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:08:12,893::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:08:12,893::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:08:12,893::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441692, 'last-update-local-ts': 1384441692.8931949, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:12,894::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:08:12,894::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441692, 'last-update-local-ts': 1384441692.8931949, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:12,894::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:08:22,982::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:08:23,240::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:08:23,331::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:08:23,392::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:08:23,393::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:08:23,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:08:23,399::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:23,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:08:23,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:08:23,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.56 >MainThread::INFO::2013-11-14 17:08:23,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:23,401::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441703|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441703 (Thu Nov 14 17:08:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.56 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:08:23,401::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:08:23,411::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:08:23,411::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:08:23,411::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441703, 'last-update-local-ts': 1384441703.4111991, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:23,412::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:08:23,412::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441703, 'last-update-local-ts': 1384441703.4111991, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:23,412::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:08:33,498::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:08:33,748::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:08:33,830::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:08:33,890::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:08:33,890::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:08:33,891::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:08:33,891::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:33,891::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:08:33,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:08:33,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.47 >MainThread::INFO::2013-11-14 17:08:33,892::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:33,893::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441713|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441713 (Thu Nov 14 17:08:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.47 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:08:33,894::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:08:33,903::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:08:33,903::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:08:33,904::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441713, 'last-update-local-ts': 1384441713.902915, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:33,904::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:08:33,904::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441713, 'last-update-local-ts': 1384441713.902915, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:33,904::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:08:43,977::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:08:44,237::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:08:44,292::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:08:44,352::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:08:44,352::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:08:44,353::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:08:44,353::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:44,353::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:08:44,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:08:44,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.4 >MainThread::INFO::2013-11-14 17:08:44,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:44,355::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441724|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441724 (Thu Nov 14 17:08:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.4 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:08:44,355::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:08:44,363::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:08:44,364::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:08:44,364::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441724, 'last-update-local-ts': 1384441724.363585, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:44,364::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:08:44,364::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441724, 'last-update-local-ts': 1384441724.363585, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:44,364::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:08:54,446::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:08:54,699::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:08:54,782::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:08:54,841::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:08:54,841::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:08:54,847::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:08:54,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:54,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:08:54,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:08:54,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.34 >MainThread::INFO::2013-11-14 17:08:54,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:08:54,849::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441734|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441734 (Thu Nov 14 17:08:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.34 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:08:54,850::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:08:54,860::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:08:54,860::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:08:54,860::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441734, 'last-update-local-ts': 1384441734.8599341, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:54,860::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:08:54,861::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441734, 'last-update-local-ts': 1384441734.8599341, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:08:54,861::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:09:04,957::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:09:05,218::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:09:05,307::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:09:05,368::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:09:05,368::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:09:05,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:09:05,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:05,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:09:05,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:09:05,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 17:09:05,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:05,371::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441745|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441745 (Thu Nov 14 17:09:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:09:05,372::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:09:05,381::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:09:05,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:09:05,382::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441745, 'last-update-local-ts': 1384441745.381259, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:05,382::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:09:05,382::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441745, 'last-update-local-ts': 1384441745.381259, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:05,382::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:09:15,477::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:09:15,751::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:09:15,842::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:09:15,903::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:09:15,903::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:09:15,904::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:09:15,904::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:15,904::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:09:15,905::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:09:15,905::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 17:09:15,905::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:15,906::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441755|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441755 (Thu Nov 14 17:09:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:09:15,906::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:09:15,914::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:09:15,914::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:09:15,915::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441755, 'last-update-local-ts': 1384441755.9143441, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:15,915::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:09:15,915::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441755, 'last-update-local-ts': 1384441755.9143441, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:15,915::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:09:25,995::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:09:26,248::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:09:26,318::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:09:26,379::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:09:26,379::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:09:26,380::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:09:26,380::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:26,380::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:09:26,380::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:09:26,381::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 17:09:26,381::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:26,382::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441766|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441766 (Thu Nov 14 17:09:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:09:26,382::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:09:26,389::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:09:26,390::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:09:26,390::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441766, 'last-update-local-ts': 1384441766.389853, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:26,390::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:09:26,390::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441765, 'last-update-local-ts': 1384441766.389853, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:26,391::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:09:36,464::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:09:36,723::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:09:36,784::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:09:36,850::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:09:36,851::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:09:36,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:09:36,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:36,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:09:36,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:09:36,852::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.175 >MainThread::INFO::2013-11-14 17:09:36,853::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:36,853::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441776|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441776 (Thu Nov 14 17:09:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.175 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:09:36,853::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:09:36,862::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:09:36,862::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:09:36,862::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441776, 'last-update-local-ts': 1384441776.8619411, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:36,862::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:09:36,862::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441776, 'last-update-local-ts': 1384441776.8619411, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:36,863::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:09:46,951::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:09:47,201::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:09:47,290::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:09:47,350::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:09:47,350::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:09:47,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:09:47,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:47,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:09:47,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:09:47,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.145 >MainThread::INFO::2013-11-14 17:09:47,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:47,359::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441787|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441787 (Thu Nov 14 17:09:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.145 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:09:47,359::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:09:47,369::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:09:47,369::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:09:47,370::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441787, 'last-update-local-ts': 1384441787.369395, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:47,370::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:09:47,370::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441786, 'last-update-local-ts': 1384441787.369395, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:47,370::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:09:57,456::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:09:57,707::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:09:57,811::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:09:57,870::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:09:57,871::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:09:57,871::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:09:57,871::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:57,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:09:57,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:09:57,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.125 >MainThread::INFO::2013-11-14 17:09:57,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:09:57,874::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441797|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441797 (Thu Nov 14 17:09:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.125 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:09:57,874::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:09:57,882::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:09:57,883::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:09:57,883::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441797, 'last-update-local-ts': 1384441797.882786, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:57,883::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:09:57,883::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441797, 'last-update-local-ts': 1384441797.882786, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:09:57,884::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:10:07,971::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:10:08,233::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:10:08,331::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:10:08,392::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:10:08,392::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:10:08,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:10:08,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:08,393::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:10:08,394::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:10:08,394::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 17:10:08,394::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:08,395::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441808|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441808 (Thu Nov 14 17:10:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:10:08,395::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:10:08,405::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:10:08,405::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:10:08,405::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441808, 'last-update-local-ts': 1384441808.405256, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:08,406::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:10:08,406::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441807, 'last-update-local-ts': 1384441808.405256, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:08,406::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:10:18,475::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:10:18,732::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:10:18,787::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:10:18,847::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:10:18,848::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:10:18,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:10:18,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:18,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:10:18,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:10:18,850::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 17:10:18,850::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:18,850::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441818|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441818 (Thu Nov 14 17:10:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:10:18,850::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:10:18,859::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:10:18,859::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:10:18,859::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441818, 'last-update-local-ts': 1384441818.8590219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:18,859::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:10:18,860::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441818, 'last-update-local-ts': 1384441818.8590219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:18,860::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:10:28,947::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:10:29,200::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:10:29,287::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:10:29,346::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:10:29,346::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:10:29,347::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:10:29,347::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:29,347::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:10:29,347::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:10:29,348::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 17:10:29,348::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:29,349::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441829|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441829 (Thu Nov 14 17:10:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:10:29,349::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:10:29,358::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:10:29,358::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:10:29,358::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441829, 'last-update-local-ts': 1384441829.3581121, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:29,358::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:10:29,359::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441828, 'last-update-local-ts': 1384441829.3581121, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:29,359::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:10:39,459::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:10:39,711::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:10:39,794::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:10:39,862::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:10:39,862::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:10:39,862::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:10:39,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:39,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:10:39,863::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:10:39,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 17:10:39,864::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:39,864::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441839|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441839 (Thu Nov 14 17:10:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:10:39,864::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:10:39,880::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:10:39,881::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:10:39,881::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441839, 'last-update-local-ts': 1384441839.8808701, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:39,881::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:10:39,881::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441839, 'last-update-local-ts': 1384441839.8808701, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:39,882::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:10:49,963::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:10:50,217::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:10:50,294::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:10:50,355::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:10:50,355::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:10:50,365::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:10:50,366::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:50,366::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:10:50,367::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:10:50,367::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 17:10:50,367::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:10:50,368::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441850|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441850 (Thu Nov 14 17:10:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:10:50,376::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:10:50,389::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:10:50,389::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:10:50,389::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441850, 'last-update-local-ts': 1384441850.389164, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:50,390::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:10:50,390::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441849, 'last-update-local-ts': 1384441850.389164, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:10:50,390::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:11:00,468::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:11:00,723::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:11:00,786::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:11:00,847::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:11:00,847::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:11:00,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:11:00,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:00,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:11:00,848::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:11:00,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 17:11:00,849::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:00,850::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441860|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441860 (Thu Nov 14 17:11:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:11:00,850::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:11:00,859::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:11:00,859::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:11:00,860::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441860, 'last-update-local-ts': 1384441860.859467, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:00,860::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:11:00,860::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441860, 'last-update-local-ts': 1384441860.859467, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:00,860::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:11:10,965::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:11:11,216::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:11:11,299::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:11:11,367::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:11:11,368::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:11:11,368::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:11:11,368::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:11,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:11:11,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:11:11,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 17:11:11,370::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:11,370::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441871|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441871 (Thu Nov 14 17:11:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:11:11,370::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:11:11,377::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:11:11,377::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:11:11,377::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441871, 'last-update-local-ts': 1384441871.37714, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:11,377::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:11:11,378::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441870, 'last-update-local-ts': 1384441871.37714, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:11,378::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:11:21,464::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:11:21,729::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:11:21,814::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:11:21,876::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:11:21,876::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:11:21,876::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:11:21,877::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:21,877::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:11:21,877::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:11:21,878::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 17:11:21,878::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:21,878::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441881|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441881 (Thu Nov 14 17:11:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:11:21,878::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:11:21,888::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:11:21,888::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:11:21,889::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441881, 'last-update-local-ts': 1384441881.888386, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:21,889::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:11:21,889::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441881, 'last-update-local-ts': 1384441881.888386, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:21,889::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:11:31,977::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:11:32,228::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:11:32,310::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:11:32,370::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:11:32,370::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:11:32,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:11:32,371::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:32,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7037 >MainThread::INFO::2013-11-14 17:11:32,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:11:32,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 17:11:32,372::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:32,373::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441892|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441892 (Thu Nov 14 17:11:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7037 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:11:32,374::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:11:32,381::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:11:32,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:11:32,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441892, 'last-update-local-ts': 1384441892.3812511, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:32,382::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:11:32,382::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441891, 'last-update-local-ts': 1384441892.3812511, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:32,382::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:11:42,460::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:11:42,724::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:11:42,806::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:11:42,869::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:11:42,869::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:11:42,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:11:42,870::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:42,870::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:11:42,870::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:11:42,870::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 17:11:42,871::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:42,871::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441902|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441902 (Thu Nov 14 17:11:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:11:42,871::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:11:42,878::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:11:42,878::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:11:42,878::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441902, 'last-update-local-ts': 1384441902.8781741, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:42,879::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:11:42,879::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441901, 'last-update-local-ts': 1384441902.8781741, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:42,879::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:11:52,956::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:11:53,221::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:11:53,280::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:11:53,339::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:11:53,339::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:11:53,346::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:11:53,346::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:53,346::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:11:53,347::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:11:53,347::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 17:11:53,347::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:11:53,348::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441913|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441913 (Thu Nov 14 17:11:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:11:53,348::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:11:53,356::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:11:53,357::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:11:53,357::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441913, 'last-update-local-ts': 1384441913.356632, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:53,357::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:11:53,357::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441912, 'last-update-local-ts': 1384441913.356632, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:11:53,357::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:12:03,452::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:12:03,700::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:12:03,794::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:12:03,855::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:12:03,856::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:12:03,856::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:12:03,856::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:03,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:12:03,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:12:03,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.02 >MainThread::INFO::2013-11-14 17:12:03,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:03,858::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441923|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441923 (Thu Nov 14 17:12:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.02 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:12:03,859::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:12:03,866::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:12:03,866::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:12:03,866::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441923, 'last-update-local-ts': 1384441923.866293, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:03,867::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:12:03,867::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441922, 'last-update-local-ts': 1384441923.866293, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:03,867::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:12:13,938::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:12:14,193::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:12:14,282::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:12:14,355::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:12:14,355::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:12:14,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:12:14,356::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:14,356::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:12:14,356::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:12:14,356::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.015 >MainThread::INFO::2013-11-14 17:12:14,357::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:14,357::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441934|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441934 (Thu Nov 14 17:12:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.015 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:12:14,357::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:12:14,365::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:12:14,366::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:12:14,366::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441934, 'last-update-local-ts': 1384441934.3658619, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:14,366::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:12:14,366::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441933, 'last-update-local-ts': 1384441934.3658619, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:14,367::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:12:24,453::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:12:24,707::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:12:24,796::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:12:24,857::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:12:24,857::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:12:24,857::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:12:24,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:24,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:12:24,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:12:24,858::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 17:12:24,859::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:24,860::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441944|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441944 (Thu Nov 14 17:12:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:12:24,861::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:12:24,871::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:12:24,871::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:12:24,872::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441944, 'last-update-local-ts': 1384441944.87135, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:24,872::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:12:24,872::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441943, 'last-update-local-ts': 1384441944.87135, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:24,872::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:12:34,949::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:12:35,198::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:12:35,281::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:12:35,343::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:12:35,343::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:12:35,344::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:12:35,344::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:35,344::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:12:35,345::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:12:35,345::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 17:12:35,345::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:35,346::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441955|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441955 (Thu Nov 14 17:12:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:12:35,346::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:12:35,354::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:12:35,354::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:12:35,355::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441955, 'last-update-local-ts': 1384441955.3544829, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:35,355::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:12:35,355::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441954, 'last-update-local-ts': 1384441955.3544829, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:35,355::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:12:45,424::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:12:45,709::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:12:45,786::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:12:45,853::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:12:45,853::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:12:45,854::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:12:45,854::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:45,855::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7032 >MainThread::INFO::2013-11-14 17:12:45,855::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:12:45,855::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 17:12:45,856::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:45,856::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441965|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441965 (Thu Nov 14 17:12:45 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7032 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:12:45,856::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:12:45,868::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:12:45,868::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:12:45,869::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441965, 'last-update-local-ts': 1384441965.868453, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:45,869::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:12:45,869::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441964, 'last-update-local-ts': 1384441965.868453, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:45,869::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:12:55,957::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:12:56,206::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:12:56,290::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:12:56,351::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:12:56,351::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:12:56,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:12:56,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:56,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:12:56,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:12:56,353::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 17:12:56,353::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:12:56,354::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441976|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441976 (Thu Nov 14 17:12:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:12:56,354::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:12:56,361::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:12:56,361::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:12:56,362::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441976, 'last-update-local-ts': 1384441976.361496, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:56,362::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:12:56,362::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441975, 'last-update-local-ts': 1384441976.361496, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:12:56,362::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:13:06,438::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:13:06,690::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:13:06,781::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:13:06,840::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:13:06,841::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:13:06,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:13:06,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:06,842::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:13:06,842::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:13:06,842::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.595 >MainThread::INFO::2013-11-14 17:13:06,842::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:06,843::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441986|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441986 (Thu Nov 14 17:13:06 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.595 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:13:06,843::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:13:06,851::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:13:06,852::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:13:06,852::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441986, 'last-update-local-ts': 1384441986.851635, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:06,852::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:13:06,852::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441985, 'last-update-local-ts': 1384441986.851635, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:06,852::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:13:16,925::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:13:17,179::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:13:17,276::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:13:17,339::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:13:17,339::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:13:17,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:13:17,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:17,340::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:13:17,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:13:17,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.505 >MainThread::INFO::2013-11-14 17:13:17,341::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:17,342::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384441997|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384441997 (Thu Nov 14 17:13:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.505 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:13:17,342::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:13:17,351::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:13:17,351::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:13:17,351::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384441997, 'last-update-local-ts': 1384441997.3511961, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:17,352::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:13:17,352::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384441995, 'last-update-local-ts': 1384441997.3511961, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:17,352::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:13:27,451::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:13:27,703::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:13:27,778::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:13:27,894::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:13:27,894::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:13:27,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:13:27,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:27,895::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:13:27,896::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:13:27,896::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.425 >MainThread::INFO::2013-11-14 17:13:27,896::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:27,897::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442007|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442007 (Thu Nov 14 17:13:27 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.425 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:13:27,897::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:13:27,906::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:13:27,906::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:13:27,907::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442007, 'last-update-local-ts': 1384442007.906249, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:27,907::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:13:27,907::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442006, 'last-update-local-ts': 1384442007.906249, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:27,907::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:13:37,988::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:13:38,248::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:13:38,311::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:13:38,371::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:13:38,371::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:13:38,377::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:13:38,378::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:38,378::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:13:38,378::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:13:38,379::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.4 >MainThread::INFO::2013-11-14 17:13:38,379::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:38,379::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442018|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442018 (Thu Nov 14 17:13:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.4 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:13:38,379::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:13:38,389::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:13:38,390::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:13:38,390::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442018, 'last-update-local-ts': 1384442018.3896911, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:38,390::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:13:38,390::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442016, 'last-update-local-ts': 1384442018.3896911, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:38,391::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:13:48,483::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:13:48,733::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:13:48,817::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:13:48,877::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:13:48,877::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:13:48,878::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:13:48,878::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:48,878::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:13:48,879::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:13:48,879::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.34 >MainThread::INFO::2013-11-14 17:13:48,879::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:48,880::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442028|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442028 (Thu Nov 14 17:13:48 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.34 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:13:48,880::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:13:48,889::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:13:48,889::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:13:48,889::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442028, 'last-update-local-ts': 1384442028.889209, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:48,890::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:13:48,890::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442027, 'last-update-local-ts': 1384442028.889209, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:48,890::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:13:58,965::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:13:59,221::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:13:59,311::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:13:59,385::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:13:59,385::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:13:59,386::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:13:59,386::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:59,386::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:13:59,386::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:13:59,387::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 17:13:59,387::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:13:59,387::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442039|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442039 (Thu Nov 14 17:13:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:13:59,388::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:13:59,396::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:13:59,396::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:13:59,396::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442039, 'last-update-local-ts': 1384442039.3961501, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:59,396::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:13:59,397::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442037, 'last-update-local-ts': 1384442039.3961501, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:13:59,397::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:14:09,478::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:14:09,731::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:14:09,811::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:14:09,872::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:14:09,872::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:14:09,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:14:09,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:09,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:14:09,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:14:09,874::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 17:14:09,874::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:09,875::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442049|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442049 (Thu Nov 14 17:14:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:14:09,875::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:14:09,884::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:14:09,885::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:14:09,885::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442049, 'last-update-local-ts': 1384442049.88484, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:09,885::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:14:09,885::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442048, 'last-update-local-ts': 1384442049.88484, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:09,886::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:14:19,960::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:14:20,215::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:14:20,296::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:14:20,359::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:14:20,359::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:14:20,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:14:20,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:20,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:14:20,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:14:20,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 17:14:20,361::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:20,361::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442060|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442060 (Thu Nov 14 17:14:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:14:20,362::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:14:20,370::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:14:20,371::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:14:20,371::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442060, 'last-update-local-ts': 1384442060.370626, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:20,371::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:14:20,371::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442058, 'last-update-local-ts': 1384442060.370626, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:20,371::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:14:30,438::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:14:30,731::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:14:30,798::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:14:30,858::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:14:30,858::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:14:30,866::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:14:30,868::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:30,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:14:30,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:14:30,869::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.725 >MainThread::INFO::2013-11-14 17:14:30,870::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:30,870::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442070|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442070 (Thu Nov 14 17:14:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.725 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:14:30,870::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:14:30,883::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:14:30,884::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:14:30,884::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442070, 'last-update-local-ts': 1384442070.8835471, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:30,884::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:14:30,884::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442069, 'last-update-local-ts': 1384442070.8835471, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:30,885::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:14:40,964::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:14:41,212::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:14:41,289::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:14:41,349::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:14:41,349::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:14:41,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:14:41,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:41,350::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:14:41,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:14:41,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.615 >MainThread::INFO::2013-11-14 17:14:41,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:41,352::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442081|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442081 (Thu Nov 14 17:14:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.615 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:14:41,353::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:14:41,361::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:14:41,361::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:14:41,362::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442081, 'last-update-local-ts': 1384442081.361367, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:41,362::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:14:41,362::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442079, 'last-update-local-ts': 1384442081.361367, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:41,362::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:14:51,440::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:14:51,687::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:14:51,770::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:14:51,830::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:14:51,831::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:14:51,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:14:51,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:51,832::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:14:51,832::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:14:51,832::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.52 >MainThread::INFO::2013-11-14 17:14:51,832::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:14:51,833::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442091|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442091 (Thu Nov 14 17:14:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.52 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:14:51,833::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:14:51,842::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:14:51,842::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:14:51,842::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442091, 'last-update-local-ts': 1384442091.8420219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:51,842::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:14:51,843::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442089, 'last-update-local-ts': 1384442091.8420219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:14:51,843::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:15:01,911::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:15:02,168::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:15:02,263::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:15:02,323::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:15:02,323::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:15:02,324::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:15:02,324::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:02,324::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:15:02,324::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:15:02,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.44 >MainThread::INFO::2013-11-14 17:15:02,325::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:02,326::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442102|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442102 (Thu Nov 14 17:15:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.44 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:15:02,326::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:15:02,335::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:15:02,336::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:15:02,336::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442102, 'last-update-local-ts': 1384442102.3356149, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:02,336::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:15:02,336::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442100, 'last-update-local-ts': 1384442102.3356149, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:02,336::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:15:12,412::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:15:12,659::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:15:12,719::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:15:12,779::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:15:12,780::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:15:12,780::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:15:12,780::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:12,781::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:15:12,781::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:15:12,781::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.37 >MainThread::INFO::2013-11-14 17:15:12,781::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:12,782::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442112|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442112 (Thu Nov 14 17:15:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.37 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:15:12,782::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:15:12,790::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:15:12,791::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:15:12,791::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442112, 'last-update-local-ts': 1384442112.7906351, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:12,791::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:15:12,791::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442110, 'last-update-local-ts': 1384442112.7906351, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:12,791::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:15:22,866::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:15:23,160::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:15:23,278::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:15:23,342::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:15:23,342::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:15:23,349::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:15:23,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:23,351::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:15:23,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:15:23,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 17:15:23,352::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:23,353::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442123|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442123 (Thu Nov 14 17:15:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:15:23,353::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:15:23,377::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:15:23,378::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:15:23,378::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442123, 'last-update-local-ts': 1384442123.3776181, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:23,378::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:15:23,378::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442121, 'last-update-local-ts': 1384442123.3776181, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:23,379::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:15:33,451::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:15:33,705::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:15:33,785::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:15:33,844::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:15:33,844::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:15:33,845::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:15:33,845::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:33,845::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:15:33,846::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:15:33,846::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 17:15:33,846::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:33,847::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442133|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442133 (Thu Nov 14 17:15:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:15:33,847::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:15:33,854::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:15:33,854::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:15:33,855::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442133, 'last-update-local-ts': 1384442133.854495, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:33,855::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:15:33,855::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442131, 'last-update-local-ts': 1384442133.854495, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:33,856::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:15:43,941::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:15:44,189::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:15:44,295::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:15:44,358::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:15:44,358::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:15:44,358::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:15:44,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:44,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:15:44,359::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:15:44,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.215 >MainThread::INFO::2013-11-14 17:15:44,360::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:44,360::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442144|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442144 (Thu Nov 14 17:15:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.215 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:15:44,360::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:15:44,367::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:15:44,367::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:15:44,367::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442144, 'last-update-local-ts': 1384442144.3672659, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:44,368::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:15:44,368::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442142, 'last-update-local-ts': 1384442144.3672659, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:44,368::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:15:54,445::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:15:54,709::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:15:54,809::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:15:54,869::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:15:54,869::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:15:54,870::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:15:54,870::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:54,870::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:15:54,871::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:15:54,871::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.625 >MainThread::INFO::2013-11-14 17:15:54,871::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:15:54,872::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442154|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442154 (Thu Nov 14 17:15:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.625 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:15:54,872::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:15:54,881::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:15:54,881::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:15:54,881::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442154, 'last-update-local-ts': 1384442154.8810101, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:54,881::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:15:54,882::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442152, 'last-update-local-ts': 1384442154.8810101, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:15:54,882::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:16:04,967::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:16:05,223::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:16:05,306::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:16:05,367::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:16:05,367::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:16:05,368::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:16:05,368::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:05,368::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:16:05,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:16:05,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.53 >MainThread::INFO::2013-11-14 17:16:05,369::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:05,370::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442165|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442165 (Thu Nov 14 17:16:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.53 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:16:05,371::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:16:05,380::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:16:05,380::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:16:05,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442165, 'last-update-local-ts': 1384442165.3805001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:05,381::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:16:05,381::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442163, 'last-update-local-ts': 1384442165.3805001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:05,381::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:16:15,457::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:16:15,716::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:16:15,792::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:16:15,872::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:16:15,872::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:16:15,872::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:16:15,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:15,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7031 >MainThread::INFO::2013-11-14 17:16:15,873::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:16:15,874::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.445 >MainThread::INFO::2013-11-14 17:16:15,874::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:15,874::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442175|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442175 (Thu Nov 14 17:16:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.445 >engine-health=vm-down >gateway=True >mem-free=7031 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:16:15,875::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:16:15,889::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:16:15,890::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:16:15,890::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442175, 'last-update-local-ts': 1384442175.8895011, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:15,890::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:16:15,890::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442173, 'last-update-local-ts': 1384442175.8895011, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:15,890::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:16:25,972::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:16:26,240::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:16:26,345::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:16:26,409::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:16:26,409::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:16:26,410::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:16:26,410::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:26,410::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:16:26,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:16:26,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.38 >MainThread::INFO::2013-11-14 17:16:26,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:26,412::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442186|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442186 (Thu Nov 14 17:16:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.38 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:16:26,412::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:16:26,421::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:16:26,422::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:16:26,422::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442186, 'last-update-local-ts': 1384442186.421607, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:26,422::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:16:26,422::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442184, 'last-update-local-ts': 1384442186.421607, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:26,423::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:16:36,517::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:16:36,766::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:16:36,858::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:16:36,918::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:16:36,919::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:16:36,919::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:16:36,919::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:36,920::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:16:36,920::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:16:36,920::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.76 >MainThread::INFO::2013-11-14 17:16:36,920::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:36,922::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442196|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442196 (Thu Nov 14 17:16:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.76 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:16:36,922::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:16:36,930::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:16:36,930::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:16:36,931::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442196, 'last-update-local-ts': 1384442196.930397, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:36,931::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:16:36,931::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442194, 'last-update-local-ts': 1384442196.930397, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:36,931::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:16:46,998::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:16:47,259::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:16:47,351::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:16:47,418::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:16:47,418::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:16:47,424::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:16:47,425::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:47,425::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:16:47,425::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:16:47,426::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.645 >MainThread::INFO::2013-11-14 17:16:47,426::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:47,426::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442207|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442207 (Thu Nov 14 17:16:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.645 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:16:47,427::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:16:47,435::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:16:47,435::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:16:47,436::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442207, 'last-update-local-ts': 1384442207.435523, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:47,436::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:16:47,436::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442205, 'last-update-local-ts': 1384442207.435523, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:47,436::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:16:57,525::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:16:57,776::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:16:57,852::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:16:57,912::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:16:57,913::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:16:57,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:16:57,913::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:57,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:16:57,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:16:57,914::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.545 >MainThread::INFO::2013-11-14 17:16:57,915::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:16:57,916::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442217|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442217 (Thu Nov 14 17:16:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.545 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:16:57,916::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:16:57,925::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:16:57,925::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:16:57,925::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442217, 'last-update-local-ts': 1384442217.92522, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:57,926::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:16:57,926::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442215, 'last-update-local-ts': 1384442217.92522, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:16:57,926::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:17:08,001::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:17:08,263::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:17:08,319::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:17:08,380::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:17:08,381::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:17:08,381::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:17:08,381::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:08,382::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:17:08,382::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:17:08,382::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.46 >MainThread::INFO::2013-11-14 17:17:08,383::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:08,383::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442228|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442228 (Thu Nov 14 17:17:08 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.46 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:17:08,383::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:17:08,391::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:17:08,392::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:17:08,392::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442228, 'last-update-local-ts': 1384442228.391804, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:08,392::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:17:08,392::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442225, 'last-update-local-ts': 1384442228.391804, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:08,393::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:17:18,481::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:17:18,728::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:17:18,817::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:17:18,877::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:17:18,877::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:17:18,884::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:17:18,884::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:18,885::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:17:18,885::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:17:18,885::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.39 >MainThread::INFO::2013-11-14 17:17:18,885::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:18,886::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442238|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442238 (Thu Nov 14 17:17:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.39 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:17:18,886::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:17:18,896::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:17:18,896::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:17:18,897::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442238, 'last-update-local-ts': 1384442238.896523, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:18,897::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:17:18,897::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442236, 'last-update-local-ts': 1384442238.896523, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:18,897::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:17:28,992::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:17:29,245::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:17:29,343::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:17:29,403::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:17:29,404::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:17:29,404::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:17:29,404::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:29,405::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:17:29,405::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:17:29,405::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.33 >MainThread::INFO::2013-11-14 17:17:29,405::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:29,406::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442249|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442249 (Thu Nov 14 17:17:29 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.33 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:17:29,407::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:17:29,415::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:17:29,416::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:17:29,416::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442249, 'last-update-local-ts': 1384442249.415653, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:29,416::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:17:29,416::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442246, 'last-update-local-ts': 1384442249.415653, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:29,416::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:17:39,498::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:17:39,753::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:17:39,841::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:17:39,900::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:17:39,901::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:17:39,907::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:17:39,907::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:39,908::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:17:39,908::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:17:39,908::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.28 >MainThread::INFO::2013-11-14 17:17:39,909::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:39,909::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442259|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442259 (Thu Nov 14 17:17:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.28 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:17:39,909::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:17:39,918::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:17:39,918::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:17:39,918::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442259, 'last-update-local-ts': 1384442259.9181061, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:39,918::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:17:39,919::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442257, 'last-update-local-ts': 1384442259.9181061, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:39,919::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:17:50,016::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:17:50,277::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:17:50,351::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:17:50,410::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:17:50,410::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:17:50,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:17:50,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:50,411::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:17:50,412::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:17:50,412::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.235 >MainThread::INFO::2013-11-14 17:17:50,412::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:17:50,413::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442270|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442270 (Thu Nov 14 17:17:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.235 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:17:50,413::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:17:50,423::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:17:50,424::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:17:50,424::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442270, 'last-update-local-ts': 1384442270.423805, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:50,424::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:17:50,424::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442267, 'last-update-local-ts': 1384442270.423805, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:17:50,425::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:18:00,501::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:18:00,860::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:18:00,923::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:18:00,984::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:18:00,984::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:18:00,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:18:00,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:00,985::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:18:00,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:18:00,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 17:18:00,986::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:00,987::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442280|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442280 (Thu Nov 14 17:18:00 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:18:00,987::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:18:00,995::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:18:00,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:18:00,996::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442280, 'last-update-local-ts': 1384442280.9956191, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:00,996::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:18:00,996::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442278, 'last-update-local-ts': 1384442280.9956191, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:00,996::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:18:11,068::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:18:11,329::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:18:11,417::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:18:11,484::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:18:11,484::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:18:11,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:18:11,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:11,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:18:11,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:18:11,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 17:18:11,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:11,493::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442291|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442291 (Thu Nov 14 17:18:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:18:11,493::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:18:11,502::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:18:11,502::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:18:11,502::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442291, 'last-update-local-ts': 1384442291.501966, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:11,502::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:18:11,503::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442288, 'last-update-local-ts': 1384442291.501966, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:11,503::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:18:21,579::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:18:21,831::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:18:21,927::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:18:21,987::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:18:21,988::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:18:21,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:18:21,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:21,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:18:21,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:18:21,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 17:18:21,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:21,991::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442301|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442301 (Thu Nov 14 17:18:21 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:18:21,991::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:18:21,999::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:18:22,000::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:18:22,000::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442301, 'last-update-local-ts': 1384442301.9997671, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:22,000::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:18:22,000::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442299, 'last-update-local-ts': 1384442301.9997671, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:22,001::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:18:32,077::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:18:32,335::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:18:32,417::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:18:32,491::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:18:32,491::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:18:32,491::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:18:32,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:32,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:18:32,492::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:18:32,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.12 >MainThread::INFO::2013-11-14 17:18:32,493::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:32,493::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442312|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442312 (Thu Nov 14 17:18:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.12 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:18:32,493::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:18:32,502::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:18:32,502::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:18:32,502::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442312, 'last-update-local-ts': 1384442312.5020981, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:32,502::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:18:32,503::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442309, 'last-update-local-ts': 1384442312.5020981, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:32,503::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:18:42,576::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:18:42,833::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:18:42,892::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:18:42,951::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:18:42,951::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:18:42,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:18:42,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:42,958::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:18:42,959::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:18:42,959::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.21 >MainThread::INFO::2013-11-14 17:18:42,959::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:42,960::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442322|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442322 (Thu Nov 14 17:18:42 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.21 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:18:42,960::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:18:42,968::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:18:42,968::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:18:42,969::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442322, 'last-update-local-ts': 1384442322.968327, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:42,969::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:18:42,969::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442319, 'last-update-local-ts': 1384442322.968327, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:42,969::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:18:53,055::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:18:53,308::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:18:53,402::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:18:53,462::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:18:53,462::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:18:53,463::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:18:53,463::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:53,463::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:18:53,464::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:18:53,464::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.15 >MainThread::INFO::2013-11-14 17:18:53,464::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:18:53,465::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442333|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442333 (Thu Nov 14 17:18:53 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.15 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:18:53,466::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:18:53,472::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:18:53,473::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:18:53,473::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442333, 'last-update-local-ts': 1384442333.472816, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:53,473::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:18:53,473::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442330, 'last-update-local-ts': 1384442333.472816, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:18:53,474::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:19:03,552::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:19:03,805::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:19:03,894::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:19:03,961::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:19:03,961::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:19:03,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:19:03,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:03,962::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:19:03,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:19:03,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.2 >MainThread::INFO::2013-11-14 17:19:03,963::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:03,964::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442343|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442343 (Thu Nov 14 17:19:03 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.2 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:19:03,964::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:19:03,971::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:19:03,971::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:19:03,971::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442343, 'last-update-local-ts': 1384442343.9709079, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:03,971::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:19:03,971::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442340, 'last-update-local-ts': 1384442343.9709079, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:03,972::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:19:14,052::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:19:14,307::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:19:14,401::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:19:14,461::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:19:14,462::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:19:14,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:19:14,468::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:14,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:19:14,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:19:14,469::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.17 >MainThread::INFO::2013-11-14 17:19:14,470::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:14,470::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442354|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442354 (Thu Nov 14 17:19:14 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.17 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:19:14,471::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:19:14,479::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:19:14,479::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:19:14,479::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442354, 'last-update-local-ts': 1384442354.4789219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:14,479::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:19:14,479::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442351, 'last-update-local-ts': 1384442354.4789219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:14,480::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:19:24,556::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:19:24,803::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:19:24,865::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:19:24,930::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:19:24,931::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:19:24,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:19:24,931::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:24,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:19:24,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:19:24,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.22 >MainThread::INFO::2013-11-14 17:19:24,932::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:24,933::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442364|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442364 (Thu Nov 14 17:19:24 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.22 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:19:24,933::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:19:24,941::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:19:24,942::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:19:24,942::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442364, 'last-update-local-ts': 1384442364.9418509, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:24,942::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:19:24,942::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442361, 'last-update-local-ts': 1384442364.9418509, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:24,943::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:19:35,014::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:19:35,313::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:19:35,422::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:19:35,487::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:19:35,487::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:19:35,495::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:19:35,495::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:35,496::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:19:35,496::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:19:35,496::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 17:19:35,497::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:35,497::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442375|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442375 (Thu Nov 14 17:19:35 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:19:35,497::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:19:35,517::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:19:35,517::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:19:35,517::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442375, 'last-update-local-ts': 1384442375.517004, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:35,517::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:19:35,518::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442372, 'last-update-local-ts': 1384442375.517004, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:35,518::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:19:45,582::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:19:45,846::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:19:45,927::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:19:46,023::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:19:46,023::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:19:46,024::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:19:46,024::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:46,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:19:46,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:19:46,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 17:19:46,025::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:46,026::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442386|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442386 (Thu Nov 14 17:19:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:19:46,026::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:19:46,067::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:19:46,067::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:19:46,067::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442386, 'last-update-local-ts': 1384442386.0668859, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:46,067::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:19:46,068::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442382, 'last-update-local-ts': 1384442386.0668859, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:46,068::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:19:56,172::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:19:56,421::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:19:56,501::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:19:56,562::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:19:56,562::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:19:56,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:19:56,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:56,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:19:56,563::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:19:56,564::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 17:19:56,564::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:19:56,565::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442396|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442396 (Thu Nov 14 17:19:56 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:19:56,565::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:19:56,572::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:19:56,572::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:19:56,572::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442396, 'last-update-local-ts': 1384442396.5722821, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:56,573::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:19:56,573::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442393, 'last-update-local-ts': 1384442396.5722821, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:19:56,573::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:20:06,657::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:20:06,915::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:20:07,050::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:20:07,130::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:20:07,130::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:20:07,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:20:07,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:07,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:20:07,131::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:20:07,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 17:20:07,132::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:07,133::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442407|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442407 (Thu Nov 14 17:20:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:20:07,133::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:20:07,142::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:20:07,142::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:20:07,144::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442407, 'last-update-local-ts': 1384442407.142482, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:07,144::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:20:07,144::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442403, 'last-update-local-ts': 1384442407.142482, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:07,144::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:20:17,236::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:20:17,496::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:20:17,594::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:20:17,656::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:20:17,656::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:20:17,657::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:20:17,657::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:17,657::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:20:17,658::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:20:17,658::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 17:20:17,658::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:17,658::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442417|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442417 (Thu Nov 14 17:20:17 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:20:17,659::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:20:17,667::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:20:17,667::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:20:17,668::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442417, 'last-update-local-ts': 1384442417.6675079, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:17,668::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:20:17,668::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442414, 'last-update-local-ts': 1384442417.6675079, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:17,668::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:20:27,742::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:20:27,998::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:20:28,077::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:20:28,137::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:20:28,137::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:20:28,138::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:20:28,138::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:28,138::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:20:28,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:20:28,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 17:20:28,139::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:28,139::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442428|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442428 (Thu Nov 14 17:20:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:20:28,140::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:20:28,148::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:20:28,148::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:20:28,148::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442428, 'last-update-local-ts': 1384442428.1483059, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:28,149::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:20:28,149::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442424, 'last-update-local-ts': 1384442428.1483059, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:28,149::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:20:38,236::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:20:38,485::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:20:38,589::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:20:38,653::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:20:38,654::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:20:38,654::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:20:38,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:38,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:20:38,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:20:38,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 17:20:38,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:38,657::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442438|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442438 (Thu Nov 14 17:20:38 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:20:38,658::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:20:38,680::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:20:38,680::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:20:38,681::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442438, 'last-update-local-ts': 1384442438.6803911, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:38,681::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:20:38,681::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442435, 'last-update-local-ts': 1384442438.6803911, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:38,681::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:20:48,756::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:20:49,009::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:20:49,092::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:20:49,151::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:20:49,152::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:20:49,152::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:20:49,152::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:49,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:20:49,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:20:49,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 17:20:49,153::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:49,154::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442449|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442449 (Thu Nov 14 17:20:49 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:20:49,154::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:20:49,164::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:20:49,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:20:49,164::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442449, 'last-update-local-ts': 1384442449.162828, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:49,164::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:20:49,165::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442445, 'last-update-local-ts': 1384442449.162828, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:49,165::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:20:59,246::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:20:59,501::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:20:59,594::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:20:59,654::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:20:59,654::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:20:59,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:20:59,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:59,655::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:20:59,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:20:59,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 17:20:59,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:20:59,657::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442459|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442459 (Thu Nov 14 17:20:59 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:20:59,658::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:20:59,667::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:20:59,667::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:20:59,668::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442459, 'last-update-local-ts': 1384442459.667383, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:59,668::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:20:59,668::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442455, 'last-update-local-ts': 1384442459.667383, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:20:59,668::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:21:09,755::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:21:10,011::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:21:10,084::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:21:10,144::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:21:10,145::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:21:10,145::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:21:10,145::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:10,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:21:10,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:21:10,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 17:21:10,146::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:10,147::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442470|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442470 (Thu Nov 14 17:21:10 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:21:10,147::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:21:10,156::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:21:10,156::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:21:10,156::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442470, 'last-update-local-ts': 1384442470.156014, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:10,156::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:21:10,157::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442466, 'last-update-local-ts': 1384442470.156014, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:10,157::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:21:20,230::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:21:20,485::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:21:20,554::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:21:20,615::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:21:20,616::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:21:20,622::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:21:20,622::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:20,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:21:20,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:21:20,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.035 >MainThread::INFO::2013-11-14 17:21:20,624::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:20,624::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442480|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442480 (Thu Nov 14 17:21:20 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.035 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:21:20,624::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:21:20,634::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:21:20,634::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:21:20,634::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442480, 'last-update-local-ts': 1384442480.6342559, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:20,635::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:21:20,635::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442476, 'last-update-local-ts': 1384442480.6342559, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:20,635::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:21:30,704::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:21:30,992::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:21:31,087::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:21:31,148::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:21:31,149::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:21:31,149::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:21:31,149::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:31,150::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:21:31,150::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:21:31,150::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 17:21:31,150::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:31,151::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442491|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442491 (Thu Nov 14 17:21:31 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:21:31,151::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:21:31,160::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:21:31,160::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:21:31,161::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442491, 'last-update-local-ts': 1384442491.160418, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:31,161::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:21:31,161::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442487, 'last-update-local-ts': 1384442491.160418, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:31,161::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:21:41,257::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:21:41,513::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:21:41,594::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:21:41,655::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:21:41,655::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:21:41,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:21:41,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:41,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:21:41,656::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:21:41,657::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 17:21:41,657::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:41,657::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442501|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442501 (Thu Nov 14 17:21:41 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:21:41,658::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:21:41,666::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:21:41,667::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:21:41,667::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442501, 'last-update-local-ts': 1384442501.6666739, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:41,667::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:21:41,667::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442497, 'last-update-local-ts': 1384442501.6666739, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:41,667::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:21:51,754::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:21:52,012::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:21:52,100::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:21:52,161::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:21:52,162::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:21:52,168::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:21:52,168::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:52,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:21:52,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:21:52,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 17:21:52,169::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:21:52,170::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442512|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442512 (Thu Nov 14 17:21:52 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:21:52,170::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:21:52,180::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:21:52,180::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:21:52,180::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442512, 'last-update-local-ts': 1384442512.1799929, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:52,180::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:21:52,181::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442508, 'last-update-local-ts': 1384442512.1799929, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:21:52,181::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:22:02,258::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:22:02,508::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:22:02,562::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:22:02,622::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:22:02,622::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:22:02,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:22:02,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:02,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:22:02,623::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:22:02,624::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.05 >MainThread::INFO::2013-11-14 17:22:02,624::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:02,625::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442522|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442522 (Thu Nov 14 17:22:02 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.05 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:22:02,625::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:22:02,634::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:22:02,634::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:22:02,634::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442522, 'last-update-local-ts': 1384442522.634239, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:02,635::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:22:02,635::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442518, 'last-update-local-ts': 1384442522.634239, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:02,635::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:22:12,725::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:22:13,034::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:22:13,141::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:22:13,206::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:22:13,206::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:22:13,206::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:22:13,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:13,207::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:22:13,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:22:13,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 17:22:13,208::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:13,209::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442533|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442533 (Thu Nov 14 17:22:13 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:22:13,209::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:22:13,220::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:22:13,220::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:22:13,221::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442533, 'last-update-local-ts': 1384442533.219887, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:13,221::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:22:13,221::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442529, 'last-update-local-ts': 1384442533.219887, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:13,222::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:22:23,307::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:22:23,573::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:22:23,647::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:22:23,708::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:22:23,708::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:22:23,715::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:22:23,715::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:23,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:22:23,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:22:23,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.065 >MainThread::INFO::2013-11-14 17:22:23,716::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:23,717::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442543|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442543 (Thu Nov 14 17:22:23 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.065 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:22:23,717::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:22:23,726::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:22:23,726::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:22:23,727::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442543, 'last-update-local-ts': 1384442543.726424, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:23,727::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:22:23,727::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442539, 'last-update-local-ts': 1384442543.726424, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:23,727::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:22:33,802::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:22:34,056::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:22:34,161::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:22:34,220::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:22:34,220::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:22:34,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:22:34,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:34,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:22:34,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:22:34,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.055 >MainThread::INFO::2013-11-14 17:22:34,222::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:34,223::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442554|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442554 (Thu Nov 14 17:22:34 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.055 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:22:34,224::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:22:34,233::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:22:34,233::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:22:34,234::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442554, 'last-update-local-ts': 1384442554.233279, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:34,234::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:22:34,234::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442549, 'last-update-local-ts': 1384442554.233279, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:34,234::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:22:44,322::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:22:44,578::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:22:44,658::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:22:44,721::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:22:44,721::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:22:44,722::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:22:44,722::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:44,722::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:22:44,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:22:44,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.045 >MainThread::INFO::2013-11-14 17:22:44,723::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:44,724::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442564|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442564 (Thu Nov 14 17:22:44 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.045 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:22:44,724::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:22:44,732::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:22:44,733::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:22:44,733::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442564, 'last-update-local-ts': 1384442564.7327919, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:44,733::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:22:44,733::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442560, 'last-update-local-ts': 1384442564.7327919, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:44,734::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:22:54,807::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:22:55,061::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:22:55,145::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:22:55,204::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:22:55,205::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:22:55,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:22:55,211::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:55,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:22:55,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:22:55,212::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.04 >MainThread::INFO::2013-11-14 17:22:55,213::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:22:55,213::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442575|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442575 (Thu Nov 14 17:22:55 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.04 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:22:55,213::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:22:55,223::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:22:55,223::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:22:55,224::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442575, 'last-update-local-ts': 1384442575.223407, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:55,224::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:22:55,224::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442570, 'last-update-local-ts': 1384442575.223407, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:22:55,224::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:23:05,322::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:23:05,572::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:23:05,648::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:23:05,709::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:23:05,709::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:23:05,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:23:05,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:05,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:23:05,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:23:05,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.03 >MainThread::INFO::2013-11-14 17:23:05,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:05,712::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442585|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442585 (Thu Nov 14 17:23:05 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.03 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:23:05,713::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:23:05,721::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:23:05,722::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:23:05,722::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442585, 'last-update-local-ts': 1384442585.7218909, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:05,722::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:23:05,722::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442581, 'last-update-local-ts': 1384442585.7218909, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:05,723::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:23:15,792::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:23:16,055::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:23:16,109::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:23:16,195::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:23:16,196::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:23:16,196::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:23:16,197::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:16,197::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7030 >MainThread::INFO::2013-11-14 17:23:16,197::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:23:16,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 17:23:16,198::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:16,198::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442596|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442596 (Thu Nov 14 17:23:16 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7030 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:23:16,198::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:23:16,216::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:23:16,217::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:23:16,217::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442596, 'last-update-local-ts': 1384442596.2164381, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:16,217::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:23:16,217::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442591, 'last-update-local-ts': 1384442596.2164381, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:16,217::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:23:26,303::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:23:26,564::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:23:26,663::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:23:26,723::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:23:26,723::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:23:26,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:23:26,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:26,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:23:26,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:23:26,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.025 >MainThread::INFO::2013-11-14 17:23:26,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:26,726::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442606|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442606 (Thu Nov 14 17:23:26 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.025 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:23:26,727::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:23:26,734::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:23:26,734::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:23:26,734::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442606, 'last-update-local-ts': 1384442606.733984, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:26,734::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:23:26,735::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442602, 'last-update-local-ts': 1384442606.733984, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:26,735::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:23:36,836::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:23:37,090::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:23:37,169::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:23:37,228::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:23:37,229::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:23:37,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:23:37,229::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:37,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:23:37,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:23:37,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.42 >MainThread::INFO::2013-11-14 17:23:37,230::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:37,231::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442617|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442617 (Thu Nov 14 17:23:37 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.42 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:23:37,231::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:23:37,240::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:23:37,240::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:23:37,240::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442617, 'last-update-local-ts': 1384442617.2399189, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:37,240::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:23:37,240::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442612, 'last-update-local-ts': 1384442617.2399189, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:37,241::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:23:47,314::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:23:47,568::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:23:47,623::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:23:47,684::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:23:47,684::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:23:47,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:23:47,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:47,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7039 >MainThread::INFO::2013-11-14 17:23:47,685::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:23:47,686::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.355 >MainThread::INFO::2013-11-14 17:23:47,686::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:47,686::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442627|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442627 (Thu Nov 14 17:23:47 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.355 >engine-health=vm-down >gateway=True >mem-free=7039 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:23:47,687::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:23:47,695::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:23:47,695::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:23:47,696::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442627, 'last-update-local-ts': 1384442627.6953781, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:47,696::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:23:47,696::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442623, 'last-update-local-ts': 1384442627.6953781, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:47,696::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:23:57,784::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:23:58,040::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:23:58,207::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:23:58,695::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:23:58,695::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:23:58,695::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:23:58,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:58,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:23:58,696::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:23:58,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.3 >MainThread::INFO::2013-11-14 17:23:58,697::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:23:58,698::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442638|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442638 (Thu Nov 14 17:23:58 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.3 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:23:58,699::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:23:58,708::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:23:58,709::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:23:58,709::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442638, 'last-update-local-ts': 1384442638.7086511, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:58,709::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:23:58,709::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442633, 'last-update-local-ts': 1384442638.7086511, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:23:58,710::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:24:08,811::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:24:09,058::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:24:09,140::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:24:09,202::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:24:09,203::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:24:09,203::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:24:09,203::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:09,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:24:09,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:24:09,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.255 >MainThread::INFO::2013-11-14 17:24:09,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:09,205::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442649|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442649 (Thu Nov 14 17:24:09 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.255 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:24:09,205::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:24:09,214::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:24:09,214::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:24:09,214::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442649, 'last-update-local-ts': 1384442649.213964, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:09,214::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:24:09,215::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442644, 'last-update-local-ts': 1384442649.213964, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:09,215::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:24:19,301::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:24:19,555::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:24:19,638::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:24:19,699::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:24:19,700::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:24:19,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:24:19,700::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:19,701::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:24:19,701::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:24:19,701::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.655 >MainThread::INFO::2013-11-14 17:24:19,701::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:19,702::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442659|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442659 (Thu Nov 14 17:24:19 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.655 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:24:19,702::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:24:19,712::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:24:19,712::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:24:19,712::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442659, 'last-update-local-ts': 1384442659.7119801, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:19,712::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:24:19,713::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442654, 'last-update-local-ts': 1384442659.7119801, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:19,713::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:24:29,784::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:24:30,123::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:24:30,213::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:24:30,274::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:24:30,275::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:24:30,275::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:24:30,275::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:30,276::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 17:24:30,276::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:24:30,276::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.555 >MainThread::INFO::2013-11-14 17:24:30,277::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:30,278::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442670|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442670 (Thu Nov 14 17:24:30 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.555 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:24:30,278::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:24:30,287::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:24:30,288::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:24:30,288::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442670, 'last-update-local-ts': 1384442670.2875831, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:30,288::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:24:30,288::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442664, 'last-update-local-ts': 1384442670.2875831, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:30,288::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:24:40,382::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:24:40,661::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:24:40,720::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:24:40,780::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:24:40,780::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:24:40,781::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:24:40,781::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:40,781::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:24:40,782::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:24:40,782::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.47 >MainThread::INFO::2013-11-14 17:24:40,782::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:40,783::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442680|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442680 (Thu Nov 14 17:24:40 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.47 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:24:40,783::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:24:40,791::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:24:40,792::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:24:40,792::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442680, 'last-update-local-ts': 1384442680.7916219, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:40,792::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:24:40,792::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442675, 'last-update-local-ts': 1384442680.7916219, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:40,792::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:24:50,867::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:24:51,146::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:24:51,217::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:24:51,277::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:24:51,277::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:24:51,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:24:51,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:51,278::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:24:51,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:24:51,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.395 >MainThread::INFO::2013-11-14 17:24:51,279::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:24:51,279::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442691|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442691 (Thu Nov 14 17:24:51 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.395 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:24:51,280::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:24:51,288::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:24:51,289::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:24:51,289::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442691, 'last-update-local-ts': 1384442691.288542, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:51,289::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:24:51,289::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442685, 'last-update-local-ts': 1384442691.288542, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:24:51,289::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:25:01,385::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:25:01,655::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:25:01,766::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:25:01,828::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:25:01,828::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:25:01,828::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:25:01,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:01,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:25:01,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:25:01,830::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.335 >MainThread::INFO::2013-11-14 17:25:01,830::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:01,831::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442701|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442701 (Thu Nov 14 17:25:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.335 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:25:01,831::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:25:01,840::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:25:01,840::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:25:01,840::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442701, 'last-update-local-ts': 1384442701.840234, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:01,841::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:25:01,841::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442696, 'last-update-local-ts': 1384442701.840234, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:01,841::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:25:11,921::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:25:12,186::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:25:12,284::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:25:12,353::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:25:12,353::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:25:12,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:25:12,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:12,354::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:25:12,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:25:12,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 17:25:12,355::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:12,355::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442712|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442712 (Thu Nov 14 17:25:12 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:25:12,356::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:25:12,364::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:25:12,364::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:25:12,365::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442712, 'last-update-local-ts': 1384442712.3643651, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:12,365::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:25:12,365::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442706, 'last-update-local-ts': 1384442712.3643651, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:12,365::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:25:22,447::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:25:22,698::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:25:22,779::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:25:22,839::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:25:22,840::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:25:22,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:25:22,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:22,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7038 >MainThread::INFO::2013-11-14 17:25:22,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:25:22,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.24 >MainThread::INFO::2013-11-14 17:25:22,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:22,843::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442722|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442722 (Thu Nov 14 17:25:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.24 >engine-health=vm-down >gateway=True >mem-free=7038 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:25:22,843::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:25:22,851::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:25:22,852::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:25:22,852::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442722, 'last-update-local-ts': 1384442722.8516979, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:22,852::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:25:22,852::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442717, 'last-update-local-ts': 1384442722.8516979, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:22,853::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:25:32,930::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:25:33,191::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:25:33,276::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:25:33,337::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:25:33,337::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:25:33,338::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:25:33,338::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:33,338::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:25:33,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:25:33,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 17:25:33,339::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:33,340::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442733|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442733 (Thu Nov 14 17:25:33 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:25:33,340::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:25:33,346::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:25:33,347::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:25:33,347::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442733, 'last-update-local-ts': 1384442733.3468909, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:33,347::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:25:33,347::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442727, 'last-update-local-ts': 1384442733.3468909, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:33,348::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:25:43,415::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:25:43,677::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:25:43,737::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:25:43,796::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:25:43,796::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:25:43,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:25:43,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:43,804::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:25:43,804::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:25:43,804::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.305 >MainThread::INFO::2013-11-14 17:25:43,805::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:43,805::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442743|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442743 (Thu Nov 14 17:25:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.305 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:25:43,805::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:25:43,814::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:25:43,814::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:25:43,814::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442743, 'last-update-local-ts': 1384442743.813976, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:43,814::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:25:43,815::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442738, 'last-update-local-ts': 1384442743.813976, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:43,815::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:25:53,901::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:25:54,147::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:25:54,243::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:25:54,304::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:25:54,305::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:25:54,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:25:54,305::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:54,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:25:54,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:25:54,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.26 >MainThread::INFO::2013-11-14 17:25:54,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:25:54,308::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442754|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442754 (Thu Nov 14 17:25:54 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.26 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:25:54,308::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:25:54,315::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:25:54,315::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:25:54,315::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442754, 'last-update-local-ts': 1384442754.3149819, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:54,315::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:25:54,316::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442748, 'last-update-local-ts': 1384442754.3149819, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:25:54,316::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:26:04,393::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:26:04,648::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:26:04,734::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:26:04,801::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:26:04,801::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:26:04,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:26:04,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:04,802::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:26:04,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:26:04,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.22 >MainThread::INFO::2013-11-14 17:26:04,803::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:04,803::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442764|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442764 (Thu Nov 14 17:26:04 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.22 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:26:04,804::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:26:04,810::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:26:04,811::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:26:04,811::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442764, 'last-update-local-ts': 1384442764.810715, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:04,811::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:26:04,811::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442758, 'last-update-local-ts': 1384442764.810715, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:04,812::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:26:14,896::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:26:15,151::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:26:15,244::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:26:15,305::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:26:15,305::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:26:15,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:26:15,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:15,306::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:26:15,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:26:15,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.185 >MainThread::INFO::2013-11-14 17:26:15,307::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:15,308::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442775|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442775 (Thu Nov 14 17:26:15 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.185 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:26:15,308::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:26:15,318::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:26:15,319::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:26:15,319::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442775, 'last-update-local-ts': 1384442775.3186131, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:15,319::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:26:15,319::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442769, 'last-update-local-ts': 1384442775.3186131, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:15,319::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:26:25,391::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:26:25,649::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:26:25,737::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:26:25,825::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:26:25,825::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:26:25,827::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:26:25,828::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:25,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:26:25,829::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:26:25,830::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.155 >MainThread::INFO::2013-11-14 17:26:25,831::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:25,832::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442785|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442785 (Thu Nov 14 17:26:25 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.155 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:26:25,832::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:26:25,843::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:26:25,849::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:26:25,851::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442785, 'last-update-local-ts': 1384442785.843811, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:25,851::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:26:25,852::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442779, 'last-update-local-ts': 1384442785.843811, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:25,854::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:26:35,944::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:26:36,237::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:26:36,412::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:26:36,487::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:26:36,487::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:26:36,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:26:36,488::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:36,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7029 >MainThread::INFO::2013-11-14 17:26:36,489::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:26:36,495::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.13 >MainThread::INFO::2013-11-14 17:26:36,495::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:36,496::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442796|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442796 (Thu Nov 14 17:26:36 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.13 >engine-health=vm-down >gateway=True >mem-free=7029 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:26:36,496::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:26:36,509::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:26:36,509::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:26:36,509::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442796, 'last-update-local-ts': 1384442796.5090671, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:36,509::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:26:36,510::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442790, 'last-update-local-ts': 1384442796.5090671, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:36,510::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:26:46,591::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:26:46,839::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:26:46,907::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:26:46,966::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:26:46,966::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:26:46,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:26:46,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:46,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:26:46,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:26:46,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.11 >MainThread::INFO::2013-11-14 17:26:46,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:46,969::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442806|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442806 (Thu Nov 14 17:26:46 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.11 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:26:46,969::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:26:46,977::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:26:46,977::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:26:46,977::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442806, 'last-update-local-ts': 1384442806.9771349, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:46,977::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:26:46,978::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442800, 'last-update-local-ts': 1384442806.9771349, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:46,978::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:26:57,075::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:26:57,328::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:26:57,410::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:26:57,478::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:26:57,478::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:26:57,479::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:26:57,479::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:57,479::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:26:57,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:26:57,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.095 >MainThread::INFO::2013-11-14 17:26:57,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:26:57,481::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442817|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442817 (Thu Nov 14 17:26:57 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.095 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:26:57,481::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:26:57,489::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:26:57,489::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:26:57,490::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442817, 'last-update-local-ts': 1384442817.489507, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:57,490::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:26:57,490::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442811, 'last-update-local-ts': 1384442817.489507, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:26:57,490::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:27:07,575::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:27:07,834::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:27:07,922::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:27:07,982::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:27:07,982::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:27:07,983::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:27:07,983::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:07,983::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:27:07,984::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:27:07,984::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.08 >MainThread::INFO::2013-11-14 17:27:07,984::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:07,985::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442827|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442827 (Thu Nov 14 17:27:07 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.08 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:27:07,985::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:27:07,994::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:27:07,995::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:27:07,995::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442827, 'last-update-local-ts': 1384442827.9946389, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:27:07,995::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:27:07,995::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442821, 'last-update-local-ts': 1384442827.9946389, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:27:07,995::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:27:18,086::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:27:18,337::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:27:18,418::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:27:18,479::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:27:18,479::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:27:18,479::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:27:18,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:18,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:27:18,480::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:27:18,481::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.105 >MainThread::INFO::2013-11-14 17:27:18,481::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:18,482::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442838|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442838 (Thu Nov 14 17:27:18 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.105 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:27:18,482::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:27:18,491::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:27:18,491::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:27:18,491::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442838, 'last-update-local-ts': 1384442838.490978, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:27:18,491::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:27:18,492::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442832, 'last-update-local-ts': 1384442838.490978, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:27:18,492::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:27:28,564::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:27:28,818::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:27:28,874::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:27:28,933::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:27:28,934::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:27:28,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:27:28,934::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:28,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7040 >MainThread::INFO::2013-11-14 17:27:28,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:27:28,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.085 >MainThread::INFO::2013-11-14 17:27:28,935::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:28,936::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442848|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442848 (Thu Nov 14 17:27:28 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.085 >engine-health=vm-down >gateway=True >mem-free=7040 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:27:28,936::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:27:28,944::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:27:28,945::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:27:28,945::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442848, 'last-update-local-ts': 1384442848.9446411, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:27:28,945::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:27:28,945::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442842, 'last-update-local-ts': 1384442848.9446411, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::ERROR::2013-11-14 17:27:28,945::hosted_engine::876::HostedEngine::(_handle_off) Engine down and local host has best score (2400), attempting to start engine VM >MainThread::INFO::2013-11-14 17:27:28,946::hosted_engine::932::HostedEngine::(_clean_vdsm_state) Ensuring VDSM state is clear for engine VM >MainThread::INFO::2013-11-14 17:27:29,000::hosted_engine::944::HostedEngine::(_clean_vdsm_state) Vdsm state for VM clean >MainThread::INFO::2013-11-14 17:27:29,000::hosted_engine::903::HostedEngine::(_start_engine_vm) Starting vm using `/usr/sbin/hosted-engine --vm-start` >MainThread::INFO::2013-11-14 17:27:29,257::hosted_engine::908::HostedEngine::(_start_engine_vm) stdout: >e4697eab-3d16-4ce6-9396-804fdbe53291 > Status = WaitForLaunch > nicModel = rtl8139,pv > emulatedMachine = rhel6.5.0 > pid = 0 > displayIp = 0 > devices = [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': '17b10cd5-13fa-4f41-b78f-10635443cfd1', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'type': 'disk', 'format': 'raw', 'bootOrder': '1', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'volumeID': '7e8c4fa6-75ab-43d2-a9e3-8644d24aaa7d', 'imageID': 'b7c0de86-7b2c-43ea-afc6-e79360de9d95', 'specParams': {}, 'readonly': 'false', 'domainID': '354a5aac-b271-49fe-bb53-7064bb1fbec2', 'deviceId': 'b7c0de86-7b2c-43ea-afc6-e79360de9d95', 'poolID': '00000000-0000-0000-0000-000000000000', 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'optional': 'false'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:17:fb:01', 'linkActive': 'true', 'network': 'rhevm', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': '33fd2981-3248-441a-9ccd-c264b45910be', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '2436cb00-2e37-4f4f-9c91-1fc6dc613abc', 'alias': 'console0'}] > smp = 2 > vmType = kvm > display = vnc > displaySecurePort = -1 > memSize = 4096 > displayPort = -1 > cpuType = Conroe > spiceSecureChannels = smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir > vmName = HostedEngine > clientIp = > >MainThread::INFO::2013-11-14 17:27:29,258::hosted_engine::909::HostedEngine::(_start_engine_vm) stderr: >MainThread::ERROR::2013-11-14 17:27:29,258::hosted_engine::922::HostedEngine::(_start_engine_vm) Engine VM started on localhost >MainThread::INFO::2013-11-14 17:27:39,369::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:27:39,629::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:27:39,771::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:27:39,839::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:27:39,840::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:27:39,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0015243902439 >MainThread::INFO::2013-11-14 17:27:39,840::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:39,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6989 >MainThread::INFO::2013-11-14 17:27:39,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:27:39,841::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.075 >MainThread::INFO::2013-11-14 17:27:39,842::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:39,842::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442859|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442859 (Thu Nov 14 17:27:39 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.075 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6989 >mem-load=0.0015243902439 > >MainThread::INFO::2013-11-14 17:27:39,843::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:27:39,855::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:27:39,856::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:27:39,856::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442859, 'last-update-local-ts': 1384442859.8556831, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:27:39,856::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:27:39,856::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442853, 'last-update-local-ts': 1384442859.8556831, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:27:39,857::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 300 seconds >MainThread::INFO::2013-11-14 17:27:49,948::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:27:50,233::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:27:50,336::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:27:50,400::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:27:50,400::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:27:50,400::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0015243902439 >MainThread::INFO::2013-11-14 17:27:50,401::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:50,401::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6974 >MainThread::INFO::2013-11-14 17:27:50,401::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:27:50,402::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.14 >MainThread::INFO::2013-11-14 17:27:50,403::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:27:50,403::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442870|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442870 (Thu Nov 14 17:27:50 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.14 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6974 >mem-load=0.0015243902439 > >MainThread::INFO::2013-11-14 17:27:50,404::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:27:50,416::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:27:50,416::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:27:50,416::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442870, 'last-update-local-ts': 1384442870.4161999, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:27:50,417::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:27:50,417::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442863, 'last-update-local-ts': 1384442870.4161999, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:27:50,417::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 289 seconds >MainThread::INFO::2013-11-14 17:28:00,585::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:28:00,901::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:28:00,988::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:28:01,056::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:28:01,056::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:28:01,060::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00228658536585 >MainThread::INFO::2013-11-14 17:28:01,061::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:01,061::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6498 >MainThread::INFO::2013-11-14 17:28:01,061::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:28:01,062::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.195 >MainThread::INFO::2013-11-14 17:28:01,062::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:01,063::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442881|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442881 (Thu Nov 14 17:28:01 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.195 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6498 >mem-load=0.00228658536585 > >MainThread::INFO::2013-11-14 17:28:01,063::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:28:01,109::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:28:01,109::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:28:01,109::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442881, 'last-update-local-ts': 1384442881.1089001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:28:01,109::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:28:01,110::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442873, 'last-update-local-ts': 1384442881.1089001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:28:01,110::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 278 seconds >MainThread::INFO::2013-11-14 17:28:11,288::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:28:11,577::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:28:11,651::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:28:11,717::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:28:11,717::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:28:11,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00241361788618 >MainThread::INFO::2013-11-14 17:28:11,727::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:11,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6452 >MainThread::INFO::2013-11-14 17:28:11,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:28:11,728::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.205 >MainThread::INFO::2013-11-14 17:28:11,729::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:11,729::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442891|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442891 (Thu Nov 14 17:28:11 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.205 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6452 >mem-load=0.00241361788618 > >MainThread::INFO::2013-11-14 17:28:11,729::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:28:11,739::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:28:11,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:28:11,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442891, 'last-update-local-ts': 1384442891.7394769, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:28:11,740::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:28:11,740::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442884, 'last-update-local-ts': 1384442891.7394769, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:28:11,740::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 268 seconds >MainThread::INFO::2013-11-14 17:28:21,854::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:28:22,130::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:28:22,242::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:28:22,331::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:28:22,332::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:28:22,332::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00241361788618 >MainThread::INFO::2013-11-14 17:28:22,333::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:22,333::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6414 >MainThread::INFO::2013-11-14 17:28:22,333::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:28:22,334::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.285 >MainThread::INFO::2013-11-14 17:28:22,334::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:22,335::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442902|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442902 (Thu Nov 14 17:28:22 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.285 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6414 >mem-load=0.00241361788618 > >MainThread::INFO::2013-11-14 17:28:22,335::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:28:22,343::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:28:22,344::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:28:22,344::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442902, 'last-update-local-ts': 1384442902.343554, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:28:22,344::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:28:22,344::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442894, 'last-update-local-ts': 1384442902.343554, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:28:22,344::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 257 seconds >MainThread::INFO::2013-11-14 17:28:32,452::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:28:32,715::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:28:32,879::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:28:32,987::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:28:32,987::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:28:32,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0025406504065 >MainThread::INFO::2013-11-14 17:28:32,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:32,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6299 >MainThread::INFO::2013-11-14 17:28:32,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:28:32,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.28 >MainThread::INFO::2013-11-14 17:28:32,989::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:32,990::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442912|1|2400|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442912 (Thu Nov 14 17:28:32 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.28 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6299 >mem-load=0.0025406504065 > >MainThread::INFO::2013-11-14 17:28:32,990::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:28:33,002::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:28:33,002::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:28:33,003::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442912, 'last-update-local-ts': 1384442913.002326, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:28:33,003::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:28:33,003::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442915, 'last-update-local-ts': 1384442913.002326, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:28:33,003::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 246 seconds >MainThread::INFO::2013-11-14 17:28:43,087::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:28:43,355::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:28:43,432::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:28:43,494::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:28:43,495::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:28:43,501::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:28:43,501::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:43,502::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7035 >MainThread::INFO::2013-11-14 17:28:43,502::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:28:43,502::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.46 >MainThread::INFO::2013-11-14 17:28:43,503::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:43,504::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442923|1|2400|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442923 (Thu Nov 14 17:28:43 2013) >host-id=1 >score=2400 >bridge=True >cpu-load=0.46 >engine-health=vm-down >gateway=True >mem-free=7035 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:28:43,504::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:28:43,515::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:28:43,515::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:28:43,515::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442923, 'last-update-local-ts': 1384442923.5151639, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-14 17:28:43,516::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:28:43,516::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442926, 'last-update-local-ts': 1384442923.5151639, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:28:43,516::hosted_engine::974::HostedEngine::(_handle_on) Engine vm died unexpectedly >MainThread::INFO::2013-11-14 17:28:53,620::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:28:53,881::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:28:53,978::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:28:54,046::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:28:54,046::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:28:54,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:28:54,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:54,047::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7035 >MainThread::INFO::2013-11-14 17:28:54,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:28:54,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.6 >MainThread::INFO::2013-11-14 17:28:54,048::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:28:54,048::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442934|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442934 (Thu Nov 14 17:28:54 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.6 >engine-health=vm-down >gateway=True >mem-free=7035 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:28:54,049::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:28:54,057::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:28:54,057::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:28:54,057::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442934, 'last-update-local-ts': 1384442934.0572579, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:28:54,058::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:28:54,058::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442936, 'last-update-local-ts': 1384442934.0572579, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:28:54,058::hosted_engine::876::HostedEngine::(_handle_off) Engine down and local host has best score (0), attempting to start engine VM >MainThread::INFO::2013-11-14 17:28:54,059::hosted_engine::932::HostedEngine::(_clean_vdsm_state) Ensuring VDSM state is clear for engine VM >MainThread::INFO::2013-11-14 17:28:54,113::hosted_engine::944::HostedEngine::(_clean_vdsm_state) Vdsm state for VM clean >MainThread::INFO::2013-11-14 17:28:54,114::hosted_engine::903::HostedEngine::(_start_engine_vm) Starting vm using `/usr/sbin/hosted-engine --vm-start` >MainThread::INFO::2013-11-14 17:28:54,416::hosted_engine::908::HostedEngine::(_start_engine_vm) stdout: >e4697eab-3d16-4ce6-9396-804fdbe53291 > Status = WaitForLaunch > nicModel = rtl8139,pv > emulatedMachine = rhel6.5.0 > pid = 0 > displayIp = 0 > devices = [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': '17b10cd5-13fa-4f41-b78f-10635443cfd1', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'type': 'disk', 'format': 'raw', 'bootOrder': '1', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'volumeID': '7e8c4fa6-75ab-43d2-a9e3-8644d24aaa7d', 'imageID': 'b7c0de86-7b2c-43ea-afc6-e79360de9d95', 'specParams': {}, 'readonly': 'false', 'domainID': '354a5aac-b271-49fe-bb53-7064bb1fbec2', 'deviceId': 'b7c0de86-7b2c-43ea-afc6-e79360de9d95', 'poolID': '00000000-0000-0000-0000-000000000000', 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'optional': 'false'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:17:fb:01', 'linkActive': 'true', 'network': 'rhevm', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': '33fd2981-3248-441a-9ccd-c264b45910be', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '2436cb00-2e37-4f4f-9c91-1fc6dc613abc', 'alias': 'console0'}] > smp = 2 > vmType = kvm > display = vnc > displaySecurePort = -1 > memSize = 4096 > displayPort = -1 > cpuType = Conroe > spiceSecureChannels = smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir > vmName = HostedEngine > clientIp = > >MainThread::INFO::2013-11-14 17:28:54,417::hosted_engine::909::HostedEngine::(_start_engine_vm) stderr: >MainThread::ERROR::2013-11-14 17:28:54,417::hosted_engine::922::HostedEngine::(_start_engine_vm) Engine VM started on localhost >MainThread::INFO::2013-11-14 17:29:04,526::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:29:04,788::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:29:04,973::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:29:05,041::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:29:05,042::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:29:05,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:29:05,042::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:05,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7033 >MainThread::INFO::2013-11-14 17:29:05,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:29:05,043::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.545 >MainThread::INFO::2013-11-14 17:29:05,044::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:05,044::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442945|1|0|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442945 (Thu Nov 14 17:29:05 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.545 >engine-health=vm-up bad-health-status >gateway=True >mem-free=7033 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:29:05,044::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:29:05,054::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:29:05,055::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:29:05,055::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442945, 'last-update-local-ts': 1384442945.0547791, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:05,055::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:29:05,055::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442947, 'last-update-local-ts': 1384442945.0547791, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:29:05,056::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 300 seconds >MainThread::INFO::2013-11-14 17:29:15,165::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:29:15,432::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:29:15,568::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:29:15,629::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:29:15,630::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:29:15,636::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0015243902439 >MainThread::INFO::2013-11-14 17:29:15,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:15,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6986 >MainThread::INFO::2013-11-14 17:29:15,637::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:29:15,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.61 >MainThread::INFO::2013-11-14 17:29:15,638::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:15,639::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442955|1|0|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442955 (Thu Nov 14 17:29:15 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.61 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6986 >mem-load=0.0015243902439 > >MainThread::INFO::2013-11-14 17:29:15,639::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:29:15,650::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:29:15,650::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:29:15,650::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442955, 'last-update-local-ts': 1384442955.6499419, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:15,650::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:29:15,650::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442957, 'last-update-local-ts': 1384442955.6499419, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::ERROR::2013-11-14 17:29:15,651::hosted_engine::1011::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 290 seconds >MainThread::INFO::2013-11-14 17:29:25,777::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:29:26,074::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:29:26,150::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:29:26,212::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:29:26,212::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:29:26,220::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.0015243902439 >MainThread::INFO::2013-11-14 17:29:26,220::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:26,220::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6774 >MainThread::INFO::2013-11-14 17:29:26,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:29:26,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.675 >MainThread::INFO::2013-11-14 17:29:26,221::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:26,222::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442966|1|0|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442966 (Thu Nov 14 17:29:26 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.675 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6774 >mem-load=0.0015243902439 > >MainThread::INFO::2013-11-14 17:29:26,222::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:29:26,238::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:29:26,239::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:29:26,239::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442966, 'last-update-local-ts': 1384442966.2386, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:26,239::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:29:26,239::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442968, 'last-update-local-ts': 1384442966.2386, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:26,240::hosted_engine::987::HostedEngine::(_handle_on) Local HA maintenance enabled >MainThread::INFO::2013-11-14 17:29:26,240::hosted_engine::1067::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-14 17:29:26,597::hosted_engine::1071::HostedEngine::(_stop_engine_vm) stdout: Machine shut down > >MainThread::INFO::2013-11-14 17:29:26,597::hosted_engine::1072::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-14 17:29:26,597::hosted_engine::1080::HostedEngine::(_stop_engine_vm) Engine VM stopped on localhost >MainThread::INFO::2013-11-14 17:29:36,703::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:29:36,982::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:29:37,117::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:29:37,186::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:29:37,187::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:29:37,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00228658536585 >MainThread::INFO::2013-11-14 17:29:37,187::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:37,188::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 6497 >MainThread::INFO::2013-11-14 17:29:37,188::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:29:37,188::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.57 >MainThread::INFO::2013-11-14 17:29:37,189::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:37,190::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442977|1|0|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442977 (Thu Nov 14 17:29:37 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.57 >engine-health=vm-up bad-health-status >gateway=True >mem-free=6497 >mem-load=0.00228658536585 > >MainThread::INFO::2013-11-14 17:29:37,190::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:29:37,199::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:29:37,200::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:29:37,200::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442977, 'last-update-local-ts': 1384442977.1996779, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:37,200::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:29:37,200::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442978, 'last-update-local-ts': 1384442977.1996779, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:37,201::hosted_engine::1067::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-14 17:29:37,473::hosted_engine::1071::HostedEngine::(_stop_engine_vm) stdout: Machine shut down > >MainThread::INFO::2013-11-14 17:29:37,473::hosted_engine::1072::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-14 17:29:37,473::hosted_engine::1080::HostedEngine::(_stop_engine_vm) Engine VM stopped on localhost >MainThread::INFO::2013-11-14 17:29:47,583::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:29:47,844::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:29:48,003::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:29:48,072::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:29:48,072::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:29:48,073::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:29:48,073::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:48,073::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7031 >MainThread::INFO::2013-11-14 17:29:48,074::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-up bad-health-status >MainThread::INFO::2013-11-14 17:29:48,074::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.52 >MainThread::INFO::2013-11-14 17:29:48,074::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:48,075::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442988|1|0|vm-up bad-health-status|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442988 (Thu Nov 14 17:29:48 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.52 >engine-health=vm-up bad-health-status >gateway=True >mem-free=7031 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:29:48,075::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:29:48,084::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:29:48,084::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:29:48,084::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442988, 'last-update-local-ts': 1384442988.084178, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:48,085::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:29:48,085::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442989, 'last-update-local-ts': 1384442988.084178, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:48,085::hosted_engine::1067::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-14 17:29:48,342::hosted_engine::1071::HostedEngine::(_stop_engine_vm) stdout: Virtual machine does not exist > >MainThread::INFO::2013-11-14 17:29:48,343::hosted_engine::1072::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-14 17:29:48,343::hosted_engine::1080::HostedEngine::(_stop_engine_vm) Engine VM stopped on localhost >MainThread::INFO::2013-11-14 17:29:58,427::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:29:58,679::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:29:58,790::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:29:58,849::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:29:58,849::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:29:58,850::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:29:58,850::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:58,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7032 >MainThread::INFO::2013-11-14 17:29:58,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:29:58,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.44 >MainThread::INFO::2013-11-14 17:29:58,851::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:29:58,852::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384442998|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384442998 (Thu Nov 14 17:29:58 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.44 >engine-health=vm-down >gateway=True >mem-free=7032 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:29:58,852::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:29:58,861::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:29:58,861::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:29:58,861::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384442998, 'last-update-local-ts': 1384442998.8609879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:58,861::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:29:58,862::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384442999, 'last-update-local-ts': 1384442998.8609879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:29:58,862::hosted_engine::1036::HostedEngine::(_handle_stop) Engine vm not running on local host >MainThread::INFO::2013-11-14 17:30:08,979::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:30:09,259::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:30:09,360::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:30:09,423::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:30:09,423::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:30:09,424::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:30:09,424::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:09,424::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7032 >MainThread::INFO::2013-11-14 17:30:09,425::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:30:09,425::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.37 >MainThread::INFO::2013-11-14 17:30:09,425::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:09,426::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443009|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443009 (Thu Nov 14 17:30:09 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.37 >engine-health=vm-down >gateway=True >mem-free=7032 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:30:09,426::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:30:09,435::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:30:09,435::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:30:09,436::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443009, 'last-update-local-ts': 1384443009.435497, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:09,436::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:30:09,436::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443010, 'last-update-local-ts': 1384443009.435497, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:09,436::hosted_engine::865::HostedEngine::(_handle_off) Local HA maintenance enabled >MainThread::INFO::2013-11-14 17:30:19,515::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:30:19,848::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:30:19,911::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:30:19,980::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:30:19,980::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:30:19,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:30:19,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:19,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7028 >MainThread::INFO::2013-11-14 17:30:19,981::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:30:19,982::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.315 >MainThread::INFO::2013-11-14 17:30:19,982::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:19,982::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443019|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443019 (Thu Nov 14 17:30:19 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.315 >engine-health=vm-down >gateway=True >mem-free=7028 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:30:19,982::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:30:19,991::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:30:19,991::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:30:19,992::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443019, 'last-update-local-ts': 1384443019.991389, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:19,992::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:30:19,992::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443020, 'last-update-local-ts': 1384443019.991389, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:19,992::hosted_engine::865::HostedEngine::(_handle_off) Local HA maintenance enabled >MainThread::INFO::2013-11-14 17:30:30,080::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:30:30,333::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:30:30,437::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:30:30,503::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:30:30,504::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:30:30,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:30:30,504::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:30,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7032 >MainThread::INFO::2013-11-14 17:30:30,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:30:30,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.265 >MainThread::INFO::2013-11-14 17:30:30,505::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:30,507::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443030|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443030 (Thu Nov 14 17:30:30 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.265 >engine-health=vm-down >gateway=True >mem-free=7032 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:30:30,507::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:30:30,516::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:30:30,516::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:30:30,517::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443030, 'last-update-local-ts': 1384443030.5163879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:30,517::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:30:30,517::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443031, 'last-update-local-ts': 1384443030.5163879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:30,517::hosted_engine::865::HostedEngine::(_handle_off) Local HA maintenance enabled >MainThread::INFO::2013-11-14 17:30:40,606::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:30:40,867::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:30:40,944::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:30:41,004::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:30:41,004::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:30:41,005::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:30:41,005::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:41,005::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7032 >MainThread::INFO::2013-11-14 17:30:41,006::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:30:41,006::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 17:30:41,006::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:41,007::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443041|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443041 (Thu Nov 14 17:30:41 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7032 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:30:41,007::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:30:41,013::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:30:41,014::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:30:41,014::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443041, 'last-update-local-ts': 1384443041.0137939, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:41,014::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:30:41,014::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443041, 'last-update-local-ts': 1384443041.0137939, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:41,015::hosted_engine::865::HostedEngine::(_handle_off) Local HA maintenance enabled >MainThread::INFO::2013-11-14 17:30:51,112::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:30:51,453::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:30:51,661::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:30:51,724::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:30:51,724::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:30:51,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:30:51,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:51,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7009 >MainThread::INFO::2013-11-14 17:30:51,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:30:51,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 17:30:51,726::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:30:51,727::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443051|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443051 (Thu Nov 14 17:30:51 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7009 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:30:51,728::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:30:51,735::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:30:51,735::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:30:51,735::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443051, 'last-update-local-ts': 1384443051.7351861, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:51,736::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:30:51,736::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443051, 'last-update-local-ts': 1384443051.7351861, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:30:51,736::hosted_engine::865::HostedEngine::(_handle_off) Local HA maintenance enabled >MainThread::INFO::2013-11-14 17:31:01,829::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::ERROR::2013-11-14 17:31:05,567::hosted_engine::343::HostedEngine::(_initialize_vdsm) Failed trying to connect storage: >MainThread::WARNING::2013-11-14 17:31:05,567::hosted_engine::262::HostedEngine::(start_monitoring) Error while monitoring engine: Failed trying to connect storage >MainThread::WARNING::2013-11-14 17:31:05,567::hosted_engine::265::HostedEngine::(start_monitoring) Unexpected error >Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 250, in start_monitoring > self._initialize_vdsm() > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 344, in _initialize_vdsm > raise Exception("Failed trying to connect storage") >Exception: Failed trying to connect storage >MainThread::INFO::2013-11-14 17:31:05,568::hosted_engine::282::HostedEngine::(start_monitoring) Sleeping 60 seconds >MainThread::ERROR::2013-11-14 17:32:05,640::hosted_engine::359::HostedEngine::(_cond_start_service) Starting vdsmd >MainThread::INFO::2013-11-14 17:32:10,333::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:32:10,504::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:32:10,566::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: NONE >MainThread::INFO::2013-11-14 17:32:10,632::hosted_engine::441::HostedEngine::(_initialize_domain_monitor) Started VDSM domain monitor for 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:32:10,695::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-14 17:32:15,762::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-14 17:32:20,826::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-14 17:32:25,890::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-14 17:32:30,966::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-14 17:32:36,027::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-14 17:32:41,091::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-14 17:32:46,155::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-14 17:32:51,218::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:32:51,218::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:32:51,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:32:51,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:32:51,225::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7056 >MainThread::INFO::2013-11-14 17:32:51,226::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:32:51,226::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.225 >MainThread::INFO::2013-11-14 17:32:51,226::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:32:51,227::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443171|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443171 (Thu Nov 14 17:32:51 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.225 >engine-health=vm-down >gateway=True >mem-free=7056 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:32:51,227::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:32:51,236::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:32:51,236::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:32:51,236::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443171, 'last-update-local-ts': 1384443171.2361791, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:32:51,237::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:32:51,237::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443205, 'last-update-local-ts': 1384443171.2361791, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:32:51,237::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:33:01,321::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:33:01,598::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:33:01,697::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:33:01,758::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:33:01,758::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:33:01,759::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:33:01,759::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:01,759::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:33:01,760::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:33:01,760::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.19 >MainThread::INFO::2013-11-14 17:33:01,760::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:01,761::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443181|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443181 (Thu Nov 14 17:33:01 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.19 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:33:01,761::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:33:01,769::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:33:01,770::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:33:01,770::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443181, 'last-update-local-ts': 1384443181.7695861, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:01,770::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:33:01,770::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443215, 'last-update-local-ts': 1384443181.7695861, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:01,771::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:33:11,863::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:33:12,111::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:33:12,194::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:33:12,256::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:33:12,256::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:33:12,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:33:12,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:12,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7056 >MainThread::INFO::2013-11-14 17:33:12,257::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:33:12,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.16 >MainThread::INFO::2013-11-14 17:33:12,258::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:12,258::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443192|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443192 (Thu Nov 14 17:33:12 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.16 >engine-health=vm-down >gateway=True >mem-free=7056 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:33:12,259::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:33:12,267::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:33:12,267::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:33:12,268::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443192, 'last-update-local-ts': 1384443192.2674971, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:12,268::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:33:12,268::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443225, 'last-update-local-ts': 1384443192.2674971, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:12,268::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:33:22,336::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:33:22,582::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:33:22,637::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:33:22,705::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:33:22,705::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:33:22,705::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:33:22,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:22,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:33:22,706::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:33:22,707::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.135 >MainThread::INFO::2013-11-14 17:33:22,707::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:22,707::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443202|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443202 (Thu Nov 14 17:33:22 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.135 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:33:22,707::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:33:22,716::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:33:22,716::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:33:22,716::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443202, 'last-update-local-ts': 1384443202.71627, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:22,717::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:33:22,717::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443236, 'last-update-local-ts': 1384443202.71627, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:22,717::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:33:32,804::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:33:33,056::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:33:33,143::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:33:33,203::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:33:33,203::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:33:33,203::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:33:33,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:33,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:33:33,204::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:33:33,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.115 >MainThread::INFO::2013-11-14 17:33:33,205::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:33,206::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443213|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443213 (Thu Nov 14 17:33:33 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.115 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:33:33,206::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:33:33,215::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:33:33,215::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:33:33,216::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443213, 'last-update-local-ts': 1384443213.2153749, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:33,216::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:33:33,216::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443246, 'last-update-local-ts': 1384443213.2153749, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up bad-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:33,216::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:33:43,317::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:33:43,572::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:33:43,656::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:33:43,723::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:33:43,724::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:33:43,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:33:43,724::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:43,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:33:43,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:33:43,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.1 >MainThread::INFO::2013-11-14 17:33:43,725::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:43,726::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443223|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443223 (Thu Nov 14 17:33:43 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.1 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:33:43,726::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:33:43,734::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:33:43,735::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:33:43,735::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443223, 'last-update-local-ts': 1384443223.734808, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:43,735::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:33:43,735::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443257, 'last-update-local-ts': 1384443223.734808, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:43,736::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:33:53,818::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:33:54,066::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:33:54,163::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:33:54,233::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:33:54,233::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:33:54,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:33:54,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:54,234::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:33:54,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:33:54,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.07 >MainThread::INFO::2013-11-14 17:33:54,235::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:33:54,236::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443234|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443234 (Thu Nov 14 17:33:54 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.07 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:33:54,237::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:33:54,245::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:33:54,246::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:33:54,246::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443234, 'last-update-local-ts': 1384443234.245806, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:54,246::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:33:54,246::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443267, 'last-update-local-ts': 1384443234.245806, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:33:54,247::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:34:04,327::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:34:04,571::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:34:04,643::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:34:04,708::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:34:04,709::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:34:04,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:34:04,709::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:04,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7054 >MainThread::INFO::2013-11-14 17:34:04,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:34:04,710::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.06 >MainThread::INFO::2013-11-14 17:34:04,711::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:04,711::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443244|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443244 (Thu Nov 14 17:34:04 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.06 >engine-health=vm-down >gateway=True >mem-free=7054 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:34:04,711::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:34:04,720::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:34:04,720::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:34:04,720::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443244, 'last-update-local-ts': 1384443244.720001, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:04,720::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:34:04,721::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443278, 'last-update-local-ts': 1384443244.720001, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:04,721::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:34:14,886::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:34:15,236::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:34:15,425::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:34:15,492::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:34:15,493::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:34:15,499::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:34:15,500::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:15,500::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7054 >MainThread::INFO::2013-11-14 17:34:15,500::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:34:15,501::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.53 >MainThread::INFO::2013-11-14 17:34:15,501::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:15,502::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443255|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443255 (Thu Nov 14 17:34:15 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.53 >engine-health=vm-down >gateway=True >mem-free=7054 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:34:15,502::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:34:15,512::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:34:15,513::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:34:15,513::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443255, 'last-update-local-ts': 1384443255.512742, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:15,513::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:34:15,513::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443288, 'last-update-local-ts': 1384443255.512742, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:15,514::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:34:25,586::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:34:25,851::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:34:25,922::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:34:25,984::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:34:25,984::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:34:25,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:34:25,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:25,991::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7041 >MainThread::INFO::2013-11-14 17:34:25,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:34:25,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.45 >MainThread::INFO::2013-11-14 17:34:25,992::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:25,993::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443265|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443265 (Thu Nov 14 17:34:25 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.45 >engine-health=vm-down >gateway=True >mem-free=7041 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:34:25,993::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:34:26,003::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:34:26,003::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:34:26,004::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443265, 'last-update-local-ts': 1384443266.0034311, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:26,004::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:34:26,004::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443299, 'last-update-local-ts': 1384443266.0034311, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:26,004::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:34:36,105::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:34:36,357::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:34:36,452::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:34:36,520::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:34:36,520::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:34:36,520::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:34:36,521::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:36,521::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7053 >MainThread::INFO::2013-11-14 17:34:36,521::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:34:36,522::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.38 >MainThread::INFO::2013-11-14 17:34:36,522::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:36,523::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443276|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443276 (Thu Nov 14 17:34:36 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.38 >engine-health=vm-down >gateway=True >mem-free=7053 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:34:36,524::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:34:36,533::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:34:36,533::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:34:36,533::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443276, 'last-update-local-ts': 1384443276.533026, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:36,533::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:34:36,534::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443309, 'last-update-local-ts': 1384443276.533026, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:36,534::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:34:46,629::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:34:46,916::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:34:47,014::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:34:47,075::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:34:47,075::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:34:47,076::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:34:47,076::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:47,076::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7050 >MainThread::INFO::2013-11-14 17:34:47,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:34:47,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.32 >MainThread::INFO::2013-11-14 17:34:47,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:47,078::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443287|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443287 (Thu Nov 14 17:34:47 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.32 >engine-health=vm-down >gateway=True >mem-free=7050 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:34:47,078::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:34:47,086::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:34:47,086::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:34:47,087::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443287, 'last-update-local-ts': 1384443287.0863831, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:47,087::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:34:47,087::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443320, 'last-update-local-ts': 1384443287.0863831, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:47,087::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:34:57,183::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:34:57,434::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:34:57,531::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:34:57,592::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:34:57,592::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:34:57,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:34:57,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:57,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7054 >MainThread::INFO::2013-11-14 17:34:57,593::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:34:57,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.75 >MainThread::INFO::2013-11-14 17:34:57,594::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:34:57,595::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443297|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443297 (Thu Nov 14 17:34:57 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.75 >engine-health=vm-down >gateway=True >mem-free=7054 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:34:57,595::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:34:57,602::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:34:57,602::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:34:57,602::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443297, 'last-update-local-ts': 1384443297.6021891, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:57,603::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:34:57,603::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443330, 'last-update-local-ts': 1384443297.6021891, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:34:57,603::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:35:07,678::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:35:07,931::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:35:08,008::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:35:08,076::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:35:08,076::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:35:08,076::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:35:08,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:08,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7054 >MainThread::INFO::2013-11-14 17:35:08,077::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:35:08,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.635 >MainThread::INFO::2013-11-14 17:35:08,078::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:08,078::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443308|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443308 (Thu Nov 14 17:35:08 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.635 >engine-health=vm-down >gateway=True >mem-free=7054 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:35:08,078::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:35:08,087::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:35:08,087::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:35:08,087::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443308, 'last-update-local-ts': 1384443308.08727, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:08,088::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:35:08,088::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443340, 'last-update-local-ts': 1384443308.08727, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:08,088::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:35:18,161::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:35:18,415::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:35:18,469::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:35:18,529::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:35:18,529::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:35:18,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:35:18,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:18,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:35:18,530::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:35:18,531::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.54 >MainThread::INFO::2013-11-14 17:35:18,531::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:18,532::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443318|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443318 (Thu Nov 14 17:35:18 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.54 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:35:18,532::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:35:18,538::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:35:18,539::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:35:18,539::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443318, 'last-update-local-ts': 1384443318.538656, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:18,539::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:35:18,539::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443351, 'last-update-local-ts': 1384443318.538656, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:18,539::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:35:28,640::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:35:28,886::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:35:28,974::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:35:29,034::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:35:29,034::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:35:29,035::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:35:29,035::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:29,035::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:35:29,036::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:35:29,036::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.49 >MainThread::INFO::2013-11-14 17:35:29,036::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:29,037::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443329|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443329 (Thu Nov 14 17:35:29 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.49 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:35:29,037::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:35:29,046::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:35:29,046::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:35:29,046::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443329, 'last-update-local-ts': 1384443329.046303, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:29,047::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:35:29,047::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443361, 'last-update-local-ts': 1384443329.046303, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:29,047::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:35:39,119::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:35:39,380::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:35:39,471::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:35:39,532::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:35:39,532::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:35:39,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:35:39,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:39,539::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:35:39,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:35:39,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.015 >MainThread::INFO::2013-11-14 17:35:39,540::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:39,541::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443339|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443339 (Thu Nov 14 17:35:39 2013) >host-id=1 >score=0 >bridge=True >cpu-load=1.015 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:35:39,541::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:35:39,549::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:35:39,549::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:35:39,549::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443339, 'last-update-local-ts': 1384443339.549123, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:39,550::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:35:39,550::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443372, 'last-update-local-ts': 1384443339.549123, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:39,550::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:35:49,630::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:35:49,881::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:35:49,953::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:35:50,015::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:35:50,016::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:35:50,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:35:50,016::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:50,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:35:50,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:35:50,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.855 >MainThread::INFO::2013-11-14 17:35:50,017::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:35:50,018::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443350|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443350 (Thu Nov 14 17:35:50 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.855 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:35:50,019::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:35:50,028::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:35:50,028::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:35:50,028::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443350, 'last-update-local-ts': 1384443350.0282259, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:50,029::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:35:50,029::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443382, 'last-update-local-ts': 1384443350.0282259, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:35:50,029::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:36:00,103::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:36:00,362::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:36:00,415::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:36:00,476::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:36:00,476::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:36:00,477::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:36:00,477::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:00,477::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:36:00,477::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:36:00,478::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.725 >MainThread::INFO::2013-11-14 17:36:00,478::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:00,478::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443360|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443360 (Thu Nov 14 17:36:00 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.725 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:36:00,479::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:36:00,487::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:36:00,487::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:36:00,488::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443360, 'last-update-local-ts': 1384443360.4873941, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:00,488::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:36:00,488::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443393, 'last-update-local-ts': 1384443360.4873941, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:00,488::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:36:10,565::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:36:10,817::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:36:10,906::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:36:10,966::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:36:10,967::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:36:10,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:36:10,967::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:10,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:36:10,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:36:10,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.615 >MainThread::INFO::2013-11-14 17:36:10,968::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:10,969::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443370|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443370 (Thu Nov 14 17:36:10 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.615 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:36:10,969::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:36:10,978::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:36:10,978::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:36:10,978::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443370, 'last-update-local-ts': 1384443370.978106, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:10,978::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:36:10,979::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443403, 'last-update-local-ts': 1384443370.978106, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:10,979::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:36:21,055::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:36:21,315::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:36:21,411::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:36:21,471::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:36:21,472::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:36:21,472::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:36:21,472::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:21,473::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:36:21,473::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:36:21,473::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 1.0 >MainThread::INFO::2013-11-14 17:36:21,474::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:21,475::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443381|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443381 (Thu Nov 14 17:36:21 2013) >host-id=1 >score=0 >bridge=True >cpu-load=1.0 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:36:21,475::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:36:21,484::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:36:21,484::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:36:21,484::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443381, 'last-update-local-ts': 1384443381.4840491, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:21,484::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:36:21,485::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443414, 'last-update-local-ts': 1384443381.4840491, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:21,485::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:36:31,571::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:36:31,830::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:36:31,923::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:36:31,986::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:36:31,986::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:36:31,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:36:31,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:31,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:36:31,987::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:36:31,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.845 >MainThread::INFO::2013-11-14 17:36:31,988::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:31,988::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443391|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443391 (Thu Nov 14 17:36:31 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.845 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:36:31,989::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:36:31,997::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:36:31,997::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:36:31,998::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443391, 'last-update-local-ts': 1384443391.9973879, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:31,998::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:36:31,998::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443424, 'last-update-local-ts': 1384443391.9973879, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:31,998::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:36:42,073::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:36:42,326::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:36:42,407::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:36:42,467::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:36:42,467::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:36:42,474::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:36:42,474::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:42,474::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:36:42,475::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:36:42,475::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.715 >MainThread::INFO::2013-11-14 17:36:42,475::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:42,476::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443402|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443402 (Thu Nov 14 17:36:42 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.715 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:36:42,476::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:36:42,485::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:36:42,485::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:36:42,486::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443402, 'last-update-local-ts': 1384443402.4853621, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:42,486::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:36:42,486::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443435, 'last-update-local-ts': 1384443402.4853621, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:42,486::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:36:52,569::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:36:52,828::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:36:52,885::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:36:52,945::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:36:52,945::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:36:52,946::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:36:52,946::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:52,946::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:36:52,947::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:36:52,947::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.605 >MainThread::INFO::2013-11-14 17:36:52,947::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:36:52,947::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443412|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443412 (Thu Nov 14 17:36:52 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.605 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:36:52,948::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:36:52,956::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:36:52,956::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:36:52,957::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443412, 'last-update-local-ts': 1384443412.9563949, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:52,957::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:36:52,957::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443445, 'last-update-local-ts': 1384443412.9563949, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:36:52,957::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:37:03,037::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:37:03,291::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:37:03,382::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:37:03,443::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:37:03,443::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:37:03,444::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:37:03,444::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:03,444::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:37:03,445::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:37:03,445::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.59 >MainThread::INFO::2013-11-14 17:37:03,445::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:03,446::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443423|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443423 (Thu Nov 14 17:37:03 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.59 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:37:03,446::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:37:03,454::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:37:03,454::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:37:03,455::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443423, 'last-update-local-ts': 1384443423.4544339, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:03,455::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:37:03,455::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443456, 'last-update-local-ts': 1384443423.4544339, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:03,455::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:37:13,536::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:37:13,788::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:37:13,882::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:37:13,942::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:37:13,942::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:37:13,943::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:37:13,943::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:13,943::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:37:13,944::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:37:13,944::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.5 >MainThread::INFO::2013-11-14 17:37:13,944::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:13,945::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443433|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443433 (Thu Nov 14 17:37:13 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.5 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:37:13,946::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:37:13,955::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:37:13,955::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:37:13,955::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443433, 'last-update-local-ts': 1384443433.954901, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:13,955::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:37:13,955::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443466, 'last-update-local-ts': 1384443433.954901, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:13,956::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:37:24,036::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:37:24,296::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:37:24,389::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:37:24,451::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:37:24,451::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:37:24,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:37:24,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:24,452::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:37:24,453::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:37:24,453::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.43 >MainThread::INFO::2013-11-14 17:37:24,453::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:24,454::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443444|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443444 (Thu Nov 14 17:37:24 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.43 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:37:24,454::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:37:24,463::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:37:24,463::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:37:24,463::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443444, 'last-update-local-ts': 1384443444.4629331, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:24,463::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:37:24,463::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443477, 'last-update-local-ts': 1384443444.4629331, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:24,464::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:37:34,550::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:37:34,801::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:37:34,888::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:37:34,947::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:37:34,948::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:37:34,948::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:37:34,948::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:34,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:37:34,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:37:34,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.365 >MainThread::INFO::2013-11-14 17:37:34,949::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:34,950::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443454|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443454 (Thu Nov 14 17:37:34 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.365 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:37:34,950::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:37:34,959::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:37:34,959::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:37:34,959::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443454, 'last-update-local-ts': 1384443454.9592521, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:34,960::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:37:34,960::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443477, 'last-update-local-ts': 1384443444.4629331, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:34,960::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:37:45,042::hosted_engine::357::HostedEngine::(_cond_start_service) vdsmd running >MainThread::INFO::2013-11-14 17:37:45,300::hosted_engine::339::HostedEngine::(_initialize_vdsm) Successfully verified that VDSM is attached to storage >MainThread::INFO::2013-11-14 17:37:45,356::hosted_engine::357::HostedEngine::(_cond_start_service) sanlock running >MainThread::INFO::2013-11-14 17:37:45,416::hosted_engine::485::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: ACQUIRED >MainThread::INFO::2013-11-14 17:37:45,416::hosted_engine::452::HostedEngine::(_initialize_domain_monitor) VDSM is monitoring domain 354a5aac-b271-49fe-bb53-7064bb1fbec2 >MainThread::INFO::2013-11-14 17:37:45,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.00139735772358 >MainThread::INFO::2013-11-14 17:37:45,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:45,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 7055 >MainThread::INFO::2013-11-14 17:37:45,417::brokerlink::125::BrokerLink::(get_monitor_status) Success, status vm-down >MainThread::INFO::2013-11-14 17:37:45,418::brokerlink::125::BrokerLink::(get_monitor_status) Success, status 0.305 >MainThread::INFO::2013-11-14 17:37:45,418::brokerlink::125::BrokerLink::(get_monitor_status) Success, status True >MainThread::INFO::2013-11-14 17:37:45,419::hosted_engine::636::HostedEngine::(_generate_local_blocks) Generated 2 blocks: >1|1|1384443465|1|0|vm-down|cyan-vdsf.qa.lab.tlv.redhat.com ><\0 padding> >metadata_parse_version=1 >metadata_feature_version=1 >timestamp=1384443465 (Thu Nov 14 17:37:45 2013) >host-id=1 >score=0 >bridge=True >cpu-load=0.305 >engine-health=vm-down >gateway=True >mem-free=7055 >mem-load=0.00139735772358 > >MainThread::INFO::2013-11-14 17:37:45,419::brokerlink::133::BrokerLink::(put_stats_on_storage) Storing blocks on storage at /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/354a5aac-b271-49fe-bb53-7064bb1fbec2/ha_agent >MainThread::INFO::2013-11-14 17:37:45,428::hosted_engine::663::HostedEngine::(_collect_all_host_stats) Global metadata: {'maintenance': False} >MainThread::INFO::2013-11-14 17:37:45,428::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) metadata updated >MainThread::INFO::2013-11-14 17:37:45,428::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1384443465, 'last-update-local-ts': 1384443465.428242, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:45,429::hosted_engine::729::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2) metadata updated >MainThread::INFO::2013-11-14 17:37:45,429::hosted_engine::734::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1384443477, 'last-update-local-ts': 1384443444.4629331, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-14 17:37:45,429::hosted_engine::855::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-14 17:37:47,975::brokerlink::79::BrokerLink::(disconnect) Closing connection to ha-broker >MainThread::INFO::2013-11-14 17:37:47,976::agent::107::Broker::(run) Agent shutting down >MainThread::INFO::2013-11-21 16:53:22,850::agent::50::Broker::(run) ovirt-hosted-engine-ha agent 0.1.0-0.6.beta1 started >MainThread::INFO::2013-11-21 16:53:22,896::hosted_engine::293::HostedEngine::(_initialize_broker) Initializing ha-broker connection >MainThread::INFO::2013-11-21 16:53:22,897::brokerlink::67::BrokerLink::(connect) Failed to connect to broker: [Errno 2] No such file or directory >MainThread::INFO::2013-11-21 16:53:22,897::brokerlink::68::BrokerLink::(connect) Retrying broker connection... >MainThread::INFO::2013-11-21 16:53:23,898::brokerlink::126::BrokerLink::(start_monitor) Starting monitor ping, options {'addr': '10.35.109.254'} >MainThread::INFO::2013-11-21 16:53:23,903::brokerlink::137::BrokerLink::(start_monitor) Success, id 38732240 >MainThread::INFO::2013-11-21 16:53:23,904::brokerlink::126::BrokerLink::(start_monitor) Starting monitor mgmt-bridge, options {'use_ssl': 'true', 'bridge_name': 'rhevm', 'address': '0'} >MainThread::INFO::2013-11-21 16:53:23,928::brokerlink::137::BrokerLink::(start_monitor) Success, id 38732880 >MainThread::INFO::2013-11-21 16:53:23,929::brokerlink::126::BrokerLink::(start_monitor) Starting monitor mem-free, options {'use_ssl': 'true', 'address': '0'} >MainThread::INFO::2013-11-21 16:53:23,932::brokerlink::137::BrokerLink::(start_monitor) Success, id 38732752 >MainThread::INFO::2013-11-21 16:53:23,933::brokerlink::126::BrokerLink::(start_monitor) Starting monitor cpu-load, options {'use_ssl': 'true', 'address': '0'} >MainThread::INFO::2013-11-21 16:53:23,938::brokerlink::137::BrokerLink::(start_monitor) Success, id 140442880679056 >MainThread::INFO::2013-11-21 16:53:23,938::brokerlink::126::BrokerLink::(start_monitor) Starting monitor mem-load, options {'use_ssl': 'true', 'address': '0'} >MainThread::INFO::2013-11-21 16:53:23,942::brokerlink::137::BrokerLink::(start_monitor) Success, id 140442880679824 >MainThread::INFO::2013-11-21 16:53:23,943::brokerlink::126::BrokerLink::(start_monitor) Starting monitor engine-health, options {'use_ssl': 'true', 'vm_uuid': '7abec612-ac58-4a8c-9558-a4470aa8ccb7', 'address': '0'} >MainThread::INFO::2013-11-21 16:53:23,957::brokerlink::137::BrokerLink::(start_monitor) Success, id 38860496 >MainThread::INFO::2013-11-21 16:53:23,957::hosted_engine::316::HostedEngine::(_initialize_broker) Broker initialized, all submonitors started >MainThread::INFO::2013-11-21 16:53:24,630::hosted_engine::384::HostedEngine::(_initialize_sanlock) Ensuring lease for lockspace hosted-engine, host id 1 is acquired (file: /rhev/data-center/mnt/lion.qa.lab.tlv.redhat.com:_export_alukiano_host__deploy/ec078415-0536-423e-a282-ab6a24d27baa/ha_agent/hosted-engine.lockspace) >MainThread::INFO::2013-11-21 16:53:45,634::hosted_engine::416::HostedEngine::(_initialize_sanlock) Acquired lock on host id 1 >MainThread::INFO::2013-11-21 16:53:45,698::hosted_engine::489::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: NONE >MainThread::INFO::2013-11-21 16:53:45,769::hosted_engine::442::HostedEngine::(_initialize_domain_monitor) Started VDSM domain monitor for ec078415-0536-423e-a282-ab6a24d27baa >MainThread::INFO::2013-11-21 16:53:45,828::hosted_engine::489::HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING >MainThread::INFO::2013-11-21 16:53:50,922::hosted_engine::741::HostedEngine::(_collect_all_host_stats) Waiting for first update from host cyan-vdsf.qa.lab.tlv.redhat.com (id 1) >MainThread::INFO::2013-11-21 16:53:50,922::hosted_engine::758::HostedEngine::(_collect_all_host_stats) Global metadata: {} >MainThread::INFO::2013-11-21 16:53:50,922::hosted_engine::762::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1385045630, 'last-update-local-ts': 1385045630, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': 'unknown', 'engine-status': 'vm-down', 'score': 2400, 'first-update': True} >MainThread::INFO::2013-11-21 16:53:50,922::hosted_engine::789::HostedEngine::(_perform_engine_actions) Unknown host state for id 1, waiting for initialization >MainThread::INFO::2013-11-21 16:54:01,434::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045641.43 type=state_transition detail=<initial_state>-ENTRY hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:54:01,436::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ENTRY) sent? ignored >MainThread::INFO::2013-11-21 16:54:01,436::hosted_engine::864::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::INFO::2013-11-21 16:54:01,436::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045641.44 type=state_transition detail=ENTRY-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:54:01,437::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ENTRY-OFF) sent? ignored >MainThread::ERROR::2013-11-21 16:54:01,437::hosted_engine::912::HostedEngine::(_handle_off) Engine down and local host has best score (2400), attempting to start engine VM >MainThread::INFO::2013-11-21 16:54:01,437::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045641.44 type=state_transition detail=OFF-START hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:54:01,642::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (OFF-START) sent? sent >MainThread::INFO::2013-11-21 16:54:01,643::hosted_engine::968::HostedEngine::(_clean_vdsm_state) Ensuring VDSM state is clear for engine VM >MainThread::INFO::2013-11-21 16:54:01,719::hosted_engine::987::HostedEngine::(_clean_vdsm_state) VM is running on host >MainThread::INFO::2013-11-21 16:54:01,719::hosted_engine::939::HostedEngine::(_start_engine_vm) Starting vm using `/usr/sbin/hosted-engine --vm-start` >MainThread::INFO::2013-11-21 16:54:01,969::hosted_engine::944::HostedEngine::(_start_engine_vm) stdout: Virtual machine already exists > >MainThread::INFO::2013-11-21 16:54:01,970::hosted_engine::945::HostedEngine::(_start_engine_vm) stderr: >MainThread::WARNING::2013-11-21 16:54:01,970::hosted_engine::951::HostedEngine::(_start_engine_vm) Failed to start engine VM, already running according to VDSM >MainThread::INFO::2013-11-21 16:54:12,655::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045652.66 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:54:12,825::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 16:54:12,825::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 300 seconds >MainThread::INFO::2013-11-21 16:54:23,484::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045663.48 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:54:23,603::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 16:54:23,604::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 289 seconds >MainThread::INFO::2013-11-21 16:54:34,362::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045674.36 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:54:34,468::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 16:54:34,469::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 278 seconds >MainThread::INFO::2013-11-21 16:54:45,046::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045685.05 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:54:45,139::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 16:54:45,140::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 267 seconds >MainThread::INFO::2013-11-21 16:54:55,708::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045695.71 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:54:55,785::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 16:54:55,786::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 257 seconds >MainThread::INFO::2013-11-21 16:55:06,556::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045706.56 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:55:06,661::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 16:55:06,661::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 246 seconds >MainThread::INFO::2013-11-21 16:55:17,400::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045717.4 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:55:17,481::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 16:55:17,482::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 235 seconds >MainThread::INFO::2013-11-21 16:55:28,065::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045728.07 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:55:28,190::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:55:28,190::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 16:55:38,843::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045738.84 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:55:38,971::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:55:49,470::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045749.47 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:55:49,541::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:56:00,087::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045760.09 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:56:00,230::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:56:10,760::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045770.76 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:56:10,859::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:56:21,392::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045781.39 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:56:21,480::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:56:32,014::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045792.01 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:56:32,080::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:56:32,081::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 16:56:42,677::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045802.68 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:56:42,813::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:56:53,344::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045813.34 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:56:53,418::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:57:03,943::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045823.94 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:57:04,006::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:57:14,771::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045834.77 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:57:14,878::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:57:25,424::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045845.42 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:57:25,490::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:57:36,043::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045856.04 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:57:36,147::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:57:36,148::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 16:57:46,608::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045866.61 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:57:46,721::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:57:57,505::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045877.51 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:57:57,593::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:58:08,159::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045888.16 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:58:08,232::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:58:19,307::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045899.31 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:58:19,464::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:58:29,937::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045909.94 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:58:30,009::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:58:40,582::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045920.58 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:58:40,674::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:58:40,675::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 16:58:51,243::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045931.24 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:58:51,321::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:59:01,841::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045941.84 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:59:01,917::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:59:12,433::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045952.43 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:59:12,515::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:59:22,992::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045962.99 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:59:23,120::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:59:33,704::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045973.7 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:59:33,787::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:59:44,798::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045984.8 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:59:45,102::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 16:59:45,102::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 16:59:55,663::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385045995.66 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 16:59:55,736::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:00:06,319::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046006.32 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:00:06,408::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:00:16,937::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046016.94 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:00:17,008::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:00:27,614::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046027.61 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:00:27,730::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:00:38,402::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046038.4 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:00:38,479::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:00:49,035::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046049.04 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:00:49,115::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:00:49,115::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:00:59,653::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046059.65 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:00:59,758::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:01:10,271::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046070.27 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:01:10,378::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:01:20,961::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046080.96 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:01:21,041::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:01:31,540::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046091.54 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:01:31,708::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:01:42,462::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046102.46 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:01:42,588::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:01:53,116::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046113.12 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:01:53,197::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:01:53,198::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:02:03,734::hosted_engine::741::HostedEngine::(_collect_all_host_stats) Waiting for first update from host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:02:03,735::hosted_engine::789::HostedEngine::(_perform_engine_actions) Unknown host state for id 2, waiting for initialization >MainThread::INFO::2013-11-21 17:02:14,260::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046134.26 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:02:14,354::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:02:25,021::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046145.02 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:02:25,117::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:02:35,633::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046155.63 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:02:35,698::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:02:46,219::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046166.22 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:02:46,304::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:02:56,851::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046176.85 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:02:56,940::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:02:56,940::hosted_engine::1052::HostedEngine::(_handle_on) Engine VM timed out with bad health status after 300 seconds, restarting >MainThread::INFO::2013-11-21 17:02:56,946::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046176.95 type=state_transition detail=ON-STOP hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:02:57,069::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ON-STOP) sent? sent >MainThread::INFO::2013-11-21 17:02:57,069::hosted_engine::1105::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-21 17:03:27,037::hosted_engine::1109::HostedEngine::(_stop_engine_vm) stdout: Failed to shutdown VM. Try Forced Shutdown. > >MainThread::INFO::2013-11-21 17:03:27,037::hosted_engine::1110::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-21 17:03:27,037::hosted_engine::1115::HostedEngine::(_stop_engine_vm) Failed to stop engine vm with /usr/sbin/hosted-engine --vm-shutdown: >MainThread::ERROR::2013-11-21 17:03:27,038::hosted_engine::1086::HostedEngine::(_handle_stop) Failed to stop engine VM: >MainThread::INFO::2013-11-21 17:03:37,614::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046217.61 type=state_transition detail=<initial_state>-ENTRY hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:03:37,615::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ENTRY) sent? ignored >MainThread::INFO::2013-11-21 17:03:37,615::hosted_engine::864::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::INFO::2013-11-21 17:03:37,616::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046217.62 type=state_transition detail=ENTRY-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:03:37,675::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ENTRY-ON) sent? sent >MainThread::ERROR::2013-11-21 17:03:37,675::hosted_engine::1034::HostedEngine::(_handle_on) Host master-vds10.qa.lab.tlv.redhat.com (id 2) score is significantly better than local score, migrating vm >MainThread::INFO::2013-11-21 17:03:37,684::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046217.68 type=state_transition detail=ON-MIGRATE hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:03:37,793::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ON-MIGRATE) sent? sent >MainThread::ERROR::2013-11-21 17:03:37,852::hosted_engine::1142::HostedEngine::(_handle_migrate) Failed to start migration >Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 1139, in _handle_migrate > dst=self._all_host_stats[best_host_id]['hostname'], > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/vds_client.py", line 85, in run_vds_client_cmd > response['status']['message']) >DetailedError: Error 4 from migrate: Virtual machine already exists >MainThread::ERROR::2013-11-21 17:03:37,854::hosted_engine::1193::HostedEngine::(_handle_migrate) Migration to host master-vds10.qa.lab.tlv.redhat.com (id 2) failed >MainThread::INFO::2013-11-21 17:03:37,854::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046217.85 type=state_transition detail=MIGRATE-STOP hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:03:37,915::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (MIGRATE-STOP) sent? sent >MainThread::INFO::2013-11-21 17:03:37,915::hosted_engine::1105::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-21 17:03:41,522::hosted_engine::1109::HostedEngine::(_stop_engine_vm) stdout: Failed to shutdown VM. Try Forced Shutdown. > >MainThread::INFO::2013-11-21 17:03:41,523::hosted_engine::1110::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-21 17:03:41,523::hosted_engine::1115::HostedEngine::(_stop_engine_vm) Failed to stop engine vm with /usr/sbin/hosted-engine --vm-shutdown: >MainThread::ERROR::2013-11-21 17:03:41,524::hosted_engine::1086::HostedEngine::(_handle_stop) Failed to stop engine VM: >MainThread::INFO::2013-11-21 17:03:52,079::hosted_engine::758::HostedEngine::(_collect_all_host_stats) Global metadata: {} >MainThread::INFO::2013-11-21 17:03:52,079::hosted_engine::762::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1385046232, 'last-update-local-ts': 1385046232, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-21 17:03:52,079::hosted_engine::762::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1385046225, 'last-update-local-ts': 1385046232, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-21 17:03:52,080::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046232.08 type=state_transition detail=<initial_state>-ENTRY hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:03:52,081::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ENTRY) sent? ignored >MainThread::INFO::2013-11-21 17:03:52,081::hosted_engine::864::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::INFO::2013-11-21 17:03:52,081::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046232.08 type=state_transition detail=ENTRY-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:03:52,082::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ENTRY-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:03:52,082::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:04:02,589::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046242.59 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:04:02,590::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:04:13,059::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046253.06 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:04:13,060::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:04:23,523::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046263.52 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:04:23,524::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:04:34,043::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046274.04 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:04:34,044::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:04:44,554::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046284.55 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:04:44,559::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:04:55,138::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046295.14 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:04:55,139::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:04:55,139::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:05:05,633::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046305.63 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:05:05,634::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:05:16,130::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046316.13 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:05:16,131::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:05:26,588::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046326.59 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:05:26,589::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:05:37,075::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046337.08 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:05:37,076::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:05:47,571::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046347.57 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:05:47,572::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:05:58,057::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046358.06 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:05:58,058::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:05:58,058::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:06:08,518::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046368.52 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:06:08,520::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:06:18,989::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046378.99 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:06:18,990::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:06:29,562::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046389.56 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:06:29,563::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:06:40,045::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046400.04 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:06:40,046::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:06:50,534::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046410.53 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:06:50,535::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:07:01,017::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046421.02 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:07:01,018::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:07:01,018::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:07:11,520::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046431.52 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:07:11,521::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:07:21,981::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046441.98 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:07:21,982::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:07:32,494::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046452.49 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:07:32,495::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:07:32,495::hosted_engine::905::HostedEngine::(_handle_off) Engine down, local host does not have best score >MainThread::INFO::2013-11-21 17:07:42,982::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046462.98 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:07:42,983::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::ERROR::2013-11-21 17:07:42,984::hosted_engine::912::HostedEngine::(_handle_off) Engine down and local host has best score (0), attempting to start engine VM >MainThread::INFO::2013-11-21 17:07:42,984::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046462.98 type=state_transition detail=OFF-START hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:07:43,093::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (OFF-START) sent? sent >MainThread::INFO::2013-11-21 17:07:43,093::hosted_engine::968::HostedEngine::(_clean_vdsm_state) Ensuring VDSM state is clear for engine VM >MainThread::INFO::2013-11-21 17:07:43,154::hosted_engine::980::HostedEngine::(_clean_vdsm_state) Vdsm state for VM clean >MainThread::INFO::2013-11-21 17:07:43,154::hosted_engine::939::HostedEngine::(_start_engine_vm) Starting vm using `/usr/sbin/hosted-engine --vm-start` >MainThread::INFO::2013-11-21 17:07:43,466::hosted_engine::944::HostedEngine::(_start_engine_vm) stdout: >7abec612-ac58-4a8c-9558-a4470aa8ccb7 > Status = WaitForLaunch > nicModel = rtl8139,pv > emulatedMachine = rhel6.5.0 > pid = 0 > displayIp = 0 > devices = [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': 'b0041622-e38d-48e6-a221-ad5d833da194', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'type': 'disk', 'format': 'raw', 'bootOrder': '1', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'volumeID': 'aa43a29e-304d-4528-96ec-15a1e4660fcc', 'imageID': '92cd2ddb-dfc3-46c8-9ee3-a9bec6b21264', 'specParams': {}, 'readonly': 'false', 'domainID': 'ec078415-0536-423e-a282-ab6a24d27baa', 'deviceId': '92cd2ddb-dfc3-46c8-9ee3-a9bec6b21264', 'poolID': '00000000-0000-0000-0000-000000000000', 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'optional': 'false'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:42:8e:f8', 'linkActive': 'true', 'network': 'rhevm', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': 'c01c4d38-721d-4438-9897-7cba7d54613d', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '9553d64e-b321-4f25-98ba-1db5fd05fc0a', 'alias': 'console0'}] > smp = 2 > vmType = kvm > display = vnc > displaySecurePort = -1 > memSize = 4096 > displayPort = -1 > cpuType = Conroe > spiceSecureChannels = smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir > vmName = HostedEngine > clientIp = > >MainThread::INFO::2013-11-21 17:07:43,466::hosted_engine::945::HostedEngine::(_start_engine_vm) stderr: >MainThread::ERROR::2013-11-21 17:07:43,466::hosted_engine::958::HostedEngine::(_start_engine_vm) Engine VM started on localhost >MainThread::INFO::2013-11-21 17:07:54,119::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046474.12 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:07:54,209::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:07:54,210::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 300 seconds >MainThread::INFO::2013-11-21 17:08:04,838::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046484.84 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:08:04,945::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:08:04,945::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 290 seconds >MainThread::INFO::2013-11-21 17:08:15,456::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046495.46 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:08:15,590::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:08:15,590::hosted_engine::1015::HostedEngine::(_handle_on) Engine vm unexpectedly running on other host >MainThread::INFO::2013-11-21 17:08:26,241::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046506.24 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:08:26,242::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:08:26,242::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:08:36,733::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046516.73 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:08:36,734::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:08:47,262::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046527.26 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:08:47,263::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:08:57,779::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046537.78 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:08:57,780::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::ERROR::2013-11-21 17:08:57,780::hosted_engine::912::HostedEngine::(_handle_off) Engine down and local host has best score (0), attempting to start engine VM >MainThread::INFO::2013-11-21 17:08:57,781::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046537.78 type=state_transition detail=OFF-START hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:08:57,885::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (OFF-START) sent? sent >MainThread::INFO::2013-11-21 17:08:57,886::hosted_engine::968::HostedEngine::(_clean_vdsm_state) Ensuring VDSM state is clear for engine VM >MainThread::INFO::2013-11-21 17:08:57,944::hosted_engine::980::HostedEngine::(_clean_vdsm_state) Vdsm state for VM clean >MainThread::INFO::2013-11-21 17:08:57,944::hosted_engine::939::HostedEngine::(_start_engine_vm) Starting vm using `/usr/sbin/hosted-engine --vm-start` >MainThread::INFO::2013-11-21 17:08:58,235::hosted_engine::944::HostedEngine::(_start_engine_vm) stdout: >7abec612-ac58-4a8c-9558-a4470aa8ccb7 > Status = WaitForLaunch > nicModel = rtl8139,pv > emulatedMachine = rhel6.5.0 > pid = 0 > displayIp = 0 > devices = [{'index': '2', 'iface': 'ide', 'specParams': {}, 'readonly': 'true', 'deviceId': 'b0041622-e38d-48e6-a221-ad5d833da194', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'type': 'disk', 'format': 'raw', 'bootOrder': '1', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'volumeID': 'aa43a29e-304d-4528-96ec-15a1e4660fcc', 'imageID': '92cd2ddb-dfc3-46c8-9ee3-a9bec6b21264', 'specParams': {}, 'readonly': 'false', 'domainID': 'ec078415-0536-423e-a282-ab6a24d27baa', 'deviceId': '92cd2ddb-dfc3-46c8-9ee3-a9bec6b21264', 'poolID': '00000000-0000-0000-0000-000000000000', 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'optional': 'false'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:42:8e:f8', 'linkActive': 'true', 'network': 'rhevm', 'filter': 'vdsm-no-mac-spoofing', 'specParams': {}, 'deviceId': 'c01c4d38-721d-4438-9897-7cba7d54613d', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId': '9553d64e-b321-4f25-98ba-1db5fd05fc0a', 'alias': 'console0'}] > smp = 2 > vmType = kvm > display = vnc > displaySecurePort = -1 > memSize = 4096 > displayPort = -1 > cpuType = Conroe > spiceSecureChannels = smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir > vmName = HostedEngine > clientIp = > >MainThread::INFO::2013-11-21 17:08:58,235::hosted_engine::945::HostedEngine::(_start_engine_vm) stderr: >MainThread::ERROR::2013-11-21 17:08:58,235::hosted_engine::958::HostedEngine::(_start_engine_vm) Engine VM started on localhost >MainThread::INFO::2013-11-21 17:09:08,861::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046548.86 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:09:08,972::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:09:08,972::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 300 seconds >MainThread::INFO::2013-11-21 17:09:19,573::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046559.57 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:09:19,702::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:09:19,702::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 289 seconds >MainThread::INFO::2013-11-21 17:09:30,260::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046570.26 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:09:30,367::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:09:30,368::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 278 seconds >MainThread::INFO::2013-11-21 17:09:40,877::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046580.88 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:09:40,995::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:09:40,995::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 268 seconds >MainThread::INFO::2013-11-21 17:09:52,019::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046592.02 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:09:52,094::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:09:52,095::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 256 seconds >MainThread::INFO::2013-11-21 17:10:02,996::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046603.0 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:10:03,266::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:10:03,266::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 245 seconds >MainThread::INFO::2013-11-21 17:10:14,075::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046614.08 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:10:14,187::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:10:14,187::hosted_engine::1047::HostedEngine::(_handle_on) Engine VM has bad health status, timeout in 234 seconds >MainThread::INFO::2013-11-21 17:10:24,935::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046624.94 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:10:25,052::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:10:25,052::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:10:36,004::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046636.0 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:10:36,194::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:10:46,796::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046646.8 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:10:46,891::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:10:57,482::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046657.48 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:10:57,556::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:11:08,236::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046668.24 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:11:08,338::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:11:18,826::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046678.83 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:11:18,922::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:11:29,479::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046689.48 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:11:29,550::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:11:29,550::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:11:40,145::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046700.15 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:11:40,248::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:11:50,768::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046710.77 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:11:50,846::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:12:01,324::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046721.32 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:12:01,406::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:12:11,968::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046731.97 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:12:12,077::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:12:22,588::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046742.59 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:12:22,666::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:12:33,220::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046753.22 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:12:33,298::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:12:33,299::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:12:43,949::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046763.95 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:12:44,237::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:12:54,721::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046774.72 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:12:54,822::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:13:05,510::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046785.51 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:13:05,630::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:13:16,159::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046796.16 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:13:16,247::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:13:26,799::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046806.8 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:13:26,871::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:13:37,371::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046817.37 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:13:37,450::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:13:37,451::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:13:48,037::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046828.04 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:13:48,136::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:13:58,687::hosted_engine::758::HostedEngine::(_collect_all_host_stats) Global metadata: {} >MainThread::INFO::2013-11-21 17:13:58,688::hosted_engine::762::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1385046838, 'last-update-local-ts': 1385046838, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-21 17:13:58,688::hosted_engine::762::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1385046828, 'last-update-local-ts': 1385046838, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-21 17:13:58,688::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046838.69 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:13:58,793::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:14:09,331::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046849.33 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:14:09,442::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:14:19,936::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046859.94 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:14:20,048::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:14:30,518::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046870.52 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:14:30,630::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:14:41,135::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046881.14 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:14:41,216::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:14:41,216::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:14:51,769::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046891.77 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:14:51,872::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:15:02,396::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046902.4 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:15:02,502::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:15:13,029::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046913.03 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:15:13,100::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:15:23,677::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046923.68 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:15:23,754::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:15:34,331::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046934.33 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:15:34,417::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:15:45,033::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046945.03 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:15:45,144::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:15:45,144::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:15:55,763::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046955.76 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:15:55,870::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:16:06,412::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046966.41 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:16:06,501::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:16:17,040::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046977.04 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:16:17,134::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:16:27,651::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046987.65 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:16:27,720::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:16:38,199::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385046998.2 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:16:38,276::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:16:48,798::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047008.8 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:16:48,872::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:16:48,873::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:16:59,417::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047019.42 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:16:59,511::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:17:10,025::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047030.03 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:17:10,108::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:17:20,585::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047040.59 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:17:20,656::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:17:31,218::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047051.22 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:17:31,343::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:17:41,888::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047061.89 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:17:41,984::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:17:52,533::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047072.53 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:17:52,622::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:17:52,623::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:18:03,153::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047083.15 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:18:03,246::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:18:13,739::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047093.74 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:18:13,828::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:18:24,430::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047104.43 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:18:24,519::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:18:35,084::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047115.08 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:18:35,164::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:18:45,798::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047125.8 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:18:45,886::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:18:56,425::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047136.43 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:18:56,491::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:18:56,491::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:19:07,052::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047147.05 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:19:07,144::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:19:17,669::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047157.67 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:19:17,779::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:19:28,311::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047168.31 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:19:28,395::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:19:38,939::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047178.94 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:19:39,037::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:19:49,550::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047189.55 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:19:49,650::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:20:00,186::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047200.19 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:20:00,280::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:20:00,280::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:20:10,768::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047210.77 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:20:10,879::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:20:21,409::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047221.41 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:20:21,492::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:20:32,003::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047232.0 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:20:32,081::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:20:42,627::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047242.63 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:20:42,717::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:20:53,191::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047253.19 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:20:53,265::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:21:03,818::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047263.82 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:21:03,911::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:21:03,911::hosted_engine::1059::HostedEngine::(_handle_on) Engine vm running on localhost >MainThread::INFO::2013-11-21 17:21:14,502::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047274.5 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:21:14,617::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::INFO::2013-11-21 17:21:25,302::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047285.3 type=state_transition detail=<initial_state>-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:21:25,394::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ON) sent? sent >MainThread::ERROR::2013-11-21 17:21:25,395::hosted_engine::1052::HostedEngine::(_handle_on) Engine VM timed out with bad health status after 300 seconds, restarting >MainThread::INFO::2013-11-21 17:21:25,424::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047285.42 type=state_transition detail=ON-STOP hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:21:25,573::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ON-STOP) sent? sent >MainThread::INFO::2013-11-21 17:21:25,574::hosted_engine::1105::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-21 17:21:55,057::hosted_engine::1109::HostedEngine::(_stop_engine_vm) stdout: Failed to shutdown VM. Try Forced Shutdown. > >MainThread::INFO::2013-11-21 17:21:55,058::hosted_engine::1110::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-21 17:21:55,058::hosted_engine::1115::HostedEngine::(_stop_engine_vm) Failed to stop engine vm with /usr/sbin/hosted-engine --vm-shutdown: >MainThread::ERROR::2013-11-21 17:21:55,065::hosted_engine::1086::HostedEngine::(_handle_stop) Failed to stop engine VM: >MainThread::INFO::2013-11-21 17:22:05,841::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047325.84 type=state_transition detail=<initial_state>-ENTRY hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:05,842::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ENTRY) sent? ignored >MainThread::INFO::2013-11-21 17:22:05,843::hosted_engine::864::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::INFO::2013-11-21 17:22:05,843::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047325.84 type=state_transition detail=ENTRY-ON hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:05,999::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ENTRY-ON) sent? sent >MainThread::ERROR::2013-11-21 17:22:05,999::hosted_engine::1034::HostedEngine::(_handle_on) Host master-vds10.qa.lab.tlv.redhat.com (id 2) score is significantly better than local score, migrating vm >MainThread::INFO::2013-11-21 17:22:06,000::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047326.0 type=state_transition detail=ON-MIGRATE hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:06,103::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ON-MIGRATE) sent? sent >MainThread::ERROR::2013-11-21 17:22:06,171::hosted_engine::1142::HostedEngine::(_handle_migrate) Failed to start migration >Traceback (most recent call last): > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 1139, in _handle_migrate > dst=self._all_host_stats[best_host_id]['hostname'], > File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/vds_client.py", line 85, in run_vds_client_cmd > response['status']['message']) >DetailedError: Error 4 from migrate: Virtual machine already exists >MainThread::ERROR::2013-11-21 17:22:06,172::hosted_engine::1193::HostedEngine::(_handle_migrate) Migration to host master-vds10.qa.lab.tlv.redhat.com (id 2) failed >MainThread::INFO::2013-11-21 17:22:06,172::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047326.17 type=state_transition detail=MIGRATE-STOP hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:06,239::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (MIGRATE-STOP) sent? sent >MainThread::INFO::2013-11-21 17:22:06,240::hosted_engine::1105::HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown` >MainThread::INFO::2013-11-21 17:22:10,639::hosted_engine::1109::HostedEngine::(_stop_engine_vm) stdout: Failed to shutdown VM. Try Forced Shutdown. > >MainThread::INFO::2013-11-21 17:22:10,640::hosted_engine::1110::HostedEngine::(_stop_engine_vm) stderr: >MainThread::ERROR::2013-11-21 17:22:10,640::hosted_engine::1115::HostedEngine::(_stop_engine_vm) Failed to stop engine vm with /usr/sbin/hosted-engine --vm-shutdown: >MainThread::ERROR::2013-11-21 17:22:10,640::hosted_engine::1086::HostedEngine::(_handle_stop) Failed to stop engine VM: >MainThread::INFO::2013-11-21 17:22:21,260::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047341.26 type=state_transition detail=<initial_state>-ENTRY hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:21,276::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-ENTRY) sent? ignored >MainThread::INFO::2013-11-21 17:22:21,276::hosted_engine::864::HostedEngine::(_handle_entry) Determining initial state for host >MainThread::INFO::2013-11-21 17:22:21,277::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047341.28 type=state_transition detail=ENTRY-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:21,277::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (ENTRY-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:22:21,278::hosted_engine::905::HostedEngine::(_handle_off) Engine down, local host does not have best score >MainThread::INFO::2013-11-21 17:22:31,799::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047351.8 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:31,800::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:22:31,800::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:22:42,311::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047362.31 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:42,312::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:22:52,847::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047372.85 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:22:52,848::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:23:03,313::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047383.31 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:23:03,314::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:23:13,815::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047393.82 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:23:13,816::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:23:24,323::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047404.32 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:23:24,324::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:23:34,814::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047414.81 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:23:34,815::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:23:34,815::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:23:45,269::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047425.27 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:23:45,270::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:23:55,841::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047435.84 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:23:55,842::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:24:06,336::hosted_engine::758::HostedEngine::(_collect_all_host_stats) Global metadata: {} >MainThread::INFO::2013-11-21 17:24:06,336::hosted_engine::762::HostedEngine::(_collect_all_host_stats) Host cyan-vdsf.qa.lab.tlv.redhat.com (id 1): {'last-update-host-ts': 1385047446, 'last-update-local-ts': 1385047446, 'hostname': 'cyan-vdsf.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-down', 'score': 0, 'first-update': False} >MainThread::INFO::2013-11-21 17:24:06,337::hosted_engine::762::HostedEngine::(_collect_all_host_stats) Host master-vds10.qa.lab.tlv.redhat.com (id 2): {'last-update-host-ts': 1385047435, 'last-update-local-ts': 1385047446, 'hostname': 'master-vds10.qa.lab.tlv.redhat.com', 'alive': True, 'engine-status': 'vm-up good-health-status', 'score': 2400, 'first-update': False} >MainThread::INFO::2013-11-21 17:24:06,337::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047446.34 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:24:06,338::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:24:16,821::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047456.82 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:24:16,822::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:24:27,321::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047467.32 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:24:27,322::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:24:37,845::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047477.85 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:24:37,846::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:24:37,846::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:24:48,306::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047488.31 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:24:48,307::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:24:58,809::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047498.81 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:24:58,810::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:25:09,394::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047509.39 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:25:09,395::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:25:19,905::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047519.91 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:25:19,906::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:25:30,397::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047530.4 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:25:30,398::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:25:40,940::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047540.94 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:25:40,941::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:25:40,941::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:25:51,440::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047551.44 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:25:51,441::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:26:01,915::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047561.92 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:26:01,916::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:26:12,404::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047572.4 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:26:12,405::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:26:22,891::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047582.89 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:26:22,892::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:26:33,377::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047593.38 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:26:33,378::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:26:43,828::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047603.83 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:26:43,829::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:26:43,829::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:26:54,374::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047614.37 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:26:54,375::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:27:04,851::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047624.85 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:27:04,852::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:27:15,342::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047635.34 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:27:15,343::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:27:25,883::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047645.88 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:27:25,886::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:27:36,414::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047656.41 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:27:36,415::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:27:46,894::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047666.89 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:27:46,895::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:27:46,895::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:27:57,360::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047677.36 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:27:57,361::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:28:07,831::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047687.83 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:28:07,832::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:28:18,352::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047698.35 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:28:18,353::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:28:28,838::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047708.84 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:28:28,839::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:28:39,296::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047719.3 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:28:39,297::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:28:49,780::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047729.78 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:28:49,781::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:28:49,782::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:29:00,239::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047740.24 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:29:00,240::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:29:10,719::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047750.72 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:29:10,720::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:29:21,211::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047761.21 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:29:21,212::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:29:31,691::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047771.69 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:29:31,693::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:29:42,143::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047782.14 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:29:42,145::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:29:52,607::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047792.61 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:29:52,608::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:29:52,608::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:30:03,149::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047803.15 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:30:03,150::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:30:13,678::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047813.68 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:30:13,679::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:30:24,158::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047824.16 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:30:24,159::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:30:34,631::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047834.63 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:30:34,632::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:30:45,087::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047845.09 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:30:45,088::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:30:55,571::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047855.57 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:30:55,572::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:30:55,572::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:31:06,071::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047866.07 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:31:06,072::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:31:16,572::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047876.57 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:31:16,573::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:31:27,065::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047887.07 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:31:27,066::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:31:37,636::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047897.64 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:31:37,637::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:31:48,105::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047908.11 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:31:48,106::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:31:58,588::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047918.59 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:31:58,589::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored >MainThread::INFO::2013-11-21 17:31:58,590::hosted_engine::889::HostedEngine::(_handle_off) Engine vm is running on host master-vds10.qa.lab.tlv.redhat.com (id 2) >MainThread::INFO::2013-11-21 17:32:09,109::brokerlink::108::BrokerLink::(notify) Trying: notify time=1385047929.11 type=state_transition detail=<initial_state>-OFF hostname='cyan-vdsf.qa.lab.tlv.redhat.com' >MainThread::INFO::2013-11-21 17:32:09,110::brokerlink::117::BrokerLink::(notify) Success, was notification of state_transition (<initial_state>-OFF) sent? ignored
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1026377
:
819182
| 827293 |
834311
|
834357